Add no-send FAQ activation evaluation
This commit is contained in:
@@ -14,6 +14,10 @@ The first version requires a top-level plain-text MIME message. HTML and multipa
|
||||
|
||||
Keep `AUTO_REPLY_ENABLE_LIVE=false` in the deployment `.env` until the Google send/reconciliation workflow and FAQ test-mode results have been accepted. Then set it true and restart both API and worker. Gmail sending must be configured, the hotel must enable staff sending, and the mailbox must have send consent. Finally, the owner explicitly confirms test-mode acceptance and selects **Enable live replies**.
|
||||
|
||||
Before test-mode acceptance, use **Batch safety evaluation** with representative synthetic cases. Each JSON case has a unique `id`, `subject`, `body`, `expectedMatch`, and optionally `expectedKnowledgeId`. The no-send evaluator accepts 1–100 cases, reads the hotel's current reviewed rules and knowledge, and reports false positives, false negatives and per-case reasons without saving a conversation or consuming a quota.
|
||||
|
||||
The minimum activation gate is zero false positives, zero false negatives for every supported exact question, and explicit exclusions covering additional requests, booking/payment/refund language, emergencies, accessibility or medical context, greetings/signatures, reply threads, and unsupported wording. Where `expectedMatch` is true, set `expectedKnowledgeId` so the case also proves the intended approved answer was selected. Keep the evaluated case set and result with the restricted acceptance record. Passing content cases does not replace mailbox-header, threading, quota, stop-control or delivery acceptance.
|
||||
|
||||
All modes default to Off. Every mode change creates a new activation boundary and invalidates previous queued automatic approvals. Only untouched messages received after activation and within the last 24 hours are eligible. Switching Test to Live does not send replies to previous test matches. The evaluation worker runs about every 30 seconds; the existing delivery worker handles approved outgoing messages.
|
||||
|
||||
## Limits and rechecks
|
||||
@@ -36,4 +40,4 @@ The database retains thread/recipient/quota reservations and evaluation evidence
|
||||
|
||||
Automated tests use fake provider handlers and MongoDB; they never send live emails. They cover exact matching, exclusions, tenant boundaries, changed answers, concurrent evaluation, quotas, rule/epoch invalidation, Gmail-thread changes and uncertain delivery. Live acceptance remains pending.
|
||||
|
||||
Test allowed questions and exclusions with your sandbox mailbox, verify duplicate prevention across restarts, inspect the actual received email, and exercise the stop control before enabling a hotel. Broad natural-language matching, multilingual questions, greetings/signature stripping, HTML/multipart equivalence and higher throughput are follow-on work requiring representative evaluation. PMS actions, payment requests and complex guest issues remain staff workflows.
|
||||
Test allowed questions and exclusions with your sandbox mailbox, verify duplicate prevention across restarts, inspect the actual received email, and exercise the stop control before enabling a hotel. Record the owner responsible for rule changes, the operator monitoring the first live window, the rollback decision-maker, and the duration of heightened monitoring. Any false positive, unexpected recipient, duplicate, uncertain unreviewed outcome, or changed knowledge answer is a stop condition: select **Turn off**, preserve evidence, and reconcile in-flight work before considering reactivation. Broad natural-language matching, multilingual questions, greetings/signature stripping, HTML/multipart equivalence and higher throughput are follow-on work requiring representative evaluation. PMS actions, payment requests and complex guest issues remain staff workflows.
|
||||
|
||||
Reference in New Issue
Block a user