Why "The Bot Works" Isn't a Metric
"The chat is working" and "the chat is working well" are different things. Without numbers, it's impossible to know whether your AI chat is worth its cost, whether the knowledge base needs improving, or how many leads you're losing to a poorly configured scenario.
Here are the key KPIs used to evaluate AI chat in business.
Metric 1: Deflection Rate
What it is: the percentage of conversations the bot resolved independently — without escalating to a human operator.
How to calculate:
Deflection Rate = (Conversations without escalation / All conversations) × 100%
Benchmark: 60–80% for a typical business. Below 50% — the bot is struggling; improve the knowledge base. Above 90% — complex issues may be getting dropped rather than escalated.
Where to find it in Auralix: Analytics → Conversations → Escalations.
Metric 2: Lead Conversion Rate
What it is: the percentage of conversations in which the user provided contact details.
How to calculate:
Lead CVR = (Conversations with contact / All conversations) × 100%
Benchmark: 5–15% for typical website traffic; 10–25% for high-intent traffic (e.g. retargeting). Below 3% — there's an issue with the timing or phrasing of the contact request.
Where to find it: Analytics → Leads.
Metric 3: First Response Time
What it is: how long it takes from the user's first message to the agent's reply.
Benchmark: under 3 seconds. An AI agent should respond instantly. If there are delays, check for plan overload or Webhook issues.
Metric 4: Containment Rate
What it is: the percentage of users who continued the conversation after the agent's first reply (didn't close the chat).
How to calculate:
Containment Rate = (Conversations with 2+ messages / All conversations) × 100%
Benchmark: 40–70%. Below 30% — the first reply isn't engaging. The bot may be too formal or off-topic.
Metric 5: Escalation Rate
What it is: the percentage of conversations handed off to a human operator.
Benchmark: 15–30%. Too high — the bot isn't coping. Too low (under 5%) — complex questions may be getting dropped without resolution rather than escalated.
Metric 6: NPS / CSAT
What it is: the user's rating after a conversation. Usually "Did this conversation help you?" with a 1–5 score.
Benchmark: CSAT 80%+ for AI chat. Below 70% — there's a systemic quality problem with the responses.
How to set it up: add a survey step at the end of your Auralix scenario.
Minimum KPI Dashboard
| KPI | Target | Alert |
|---|---|---|
| Deflection Rate | > 65% | < 50% |
| Lead CVR | > 8% | < 3% |
| Containment Rate | > 50% | < 30% |
| Escalation Rate | 15–25% | > 40% or < 5% |
| First Response | < 3 sec | > 5 sec |
FAQ
How often should I review these metrics? Daily for the first 2 weeks. After that, weekly and whenever you make changes (new promotion, updated knowledge base).
What should I do if the Deflection Rate is low? Analyse which questions are triggering escalations — then add answers to those questions in the knowledge base.
Can I measure ROI from AI chat? Yes: (Operator hourly cost × Hours saved by the bot) − Auralix subscription cost. Most customers reach payback within 1–2 months.
Summary
Metrics aren't bureaucracy — they're an improvement tool. Deflection rate shows knowledge base quality, lead CVR shows scenario quality, NPS shows answer quality. Check the numbers weekly and iterate.
