
Short answer: AI quality is controlled by four layers: answers sourced only from your knowledge base, a test question set, monitoring of real conversations, and honest escalation to humans. Setup from $1,000.
The number one fear executives have about AI adoption is «it will tell customers nonsense». The fear is justified if the assistant answers from the model’s «head». In a correct architecture it answers only with facts from your documents — and quality becomes manageable.
Four layers of control
- Knowledge base as the only source: the AI finds the answer in your documents and quotes it instead of improvising;
- Test set: 50–200 reference «question — correct answer» pairs; every assistant update is run through it;
- Conversation monitoring: a weekly sample of real dialogues, error analysis, knowledge-base updates;
- Escalation: at low confidence the assistant honestly hands the conversation to a human — no invented answers.
Quality metrics
Share of answers with a found source, escalation rate, customer dialogue ratings, error rate on the test set after updates. You see these numbers in a regular report. The technical foundation is document search via AIHub; related: ChatGPT vs on-premise AI and our IT consulting service.
FAQ
Can AI errors be eliminated completely?
No — as with humans. But they can be made rare, visible and safe: no invented facts, with handover to a person.
What are AI hallucinations?
A confident but fabricated answer. The cure is architectural: answers only from the knowledge base, with source citation.
Who watches quality after launch?
We do — as part of maintenance: conversation monitoring, test and knowledge-base updates, metric reports.
How much does quality-control setup cost?
From $1,000 for an existing assistant: audit, test set, monitoring.