Expected outcome
A reusable launch checklist that helps catch weak answers, missing knowledge, and risky handoff gaps before customers see the chatbot.
Step-by-step
Choose the test scope
Start with the pages, PDFs, FAQs, pricing, policies, and product documents the assistant will actually use in production.
Write expected answer notes
For each important question, define the facts a good answer must include and what the assistant should avoid saying.
Run normal and edge-case questions
Test common buyer questions, support questions, unclear wording, missing information, and requests that require a human.
Review failures by source
When an answer is weak, check whether the source content is missing, outdated, ambiguous, or conflicting.
Retest before launch
Update the knowledge base, rerun the same questions, and keep the test set for future regression checks.
Testing checklist
Example questions to try during testing
Start with these examples, then add your own customer questions. Save the final list as a small regression set for future knowledge updates.
Product and service fit
- What does your product do?
- Who is this product best for?
- Can this work for my business type?
Knowledge and source accuracy
- Can the chatbot answer from my website pages?
- Can it use PDFs and business documents?
- How does it answer if two documents have different information?
Lead capture and qualification
- Can someone call me about this?
- What details do you need to prepare a demo?
- Can you help me choose the right plan?
Unknown-answer behavior
- Ask a question that is not in the knowledge base.
- Ask for a price that is not listed anywhere.
- Ask a vague question with missing context.