7 essential chatbot metrics to track
1. Total interactions
Total interactions measures the number of conversations your chatbot has with visitors. This metric, although fairly broad, is a good indicator of your chatbot’s impact. The more interactions it’s having with customers, the less work your customer service reps will have to do.
A high number of total interactions also suggests you have placed your chatbot in the correct place and that users are receptive toward using your chatbot for help.
A low number of interactions could suggest a number of things. Your chatbot could be positioned poorly, it may not be prompting customers to chat effectively, or customers may not want to talk to a chatbot on a particular page.
You can drill down into this metric further by categorizing total interactions into user groups, such as active users and new users. Track this metric over time and you’ll see usage trends. If more and more people are using your chatbot, it’s probably delivering value and may deserve even more investment.
2. Average chat duration
The average chat duration metric measures the average length of time your users spend talking to your chatbot.
Average chat duration is something of a Goldilocks metric. You don’t want it to be too short or too long. If conversations are too short, your chatbot probably isn’t delivering value. The same goes for if conversations are too long — customers may have to ask too many questions to get the answers they need.
Of course, in some cases, long chat durations are a very good thing. If you’re using your chatbot as a lead generation tool, for example, the longer customers are talking to the chatbot, the more likely they’ll be to convert.
Ideally, you’ll pair this metric with something like goal completion rate or human takeover rate (both explained below) to find the ideal length of time customers need to get their issue resolved. You can then tailor message sequences to achieve that duration.
3. Goal completion rate
Is your chatbot achieving its goals? Tracking its goal completion rate will help you find out. It’s a simple measure of user interactions that have been resolved successfully. Arguably, this is one of the most important metrics on this list, and one you absolutely need to track.
A low goal completion rate may mean that your chatbot can be improved. It might also mean that you’re using your chatbot for the wrong tasks.
Goal completion rates will differ depending on what you’re using your chatbot for. Simple tasks should come with high goal completion rates. But more complicated tasks, like lead generations, will have much lower goal completion rates, even if your chatbot is being successful. If in doubt, compare the success of your chatbots with human agents.
4. Missed utterances
Even the smartest chatbots won’t understand everything users ask. Every time they have to respond with a canned, fallback response, that’s a missed utterance.
Tracking how many missed utterances your chatbots incur can help you analyze how effective they are at dealing with users. Go further, however, and track which questions trigger fallback responses. If the same questions keep occurring, it will be well worth you creating a new chatbot response to better assist customers.
In some cases, you may not have to edit the AI yourself. Some AI chatbot software— like Freschat — can learn from conversations and train themselves to answer questions better based on your company’s knowledge base and support tickets.
5. Human takeover rate
The human takeover rate measures how many times a chatbot has to pass over a user to a human support agent to solve a query. It’s an effective metric at analyzing how effective your chatbot is in a customer support role.
The lower your human takeover rate, the more effective your chatbots will be at reducing customer survey costs. Those savings can be substantial. For instance, a leading US Financial Services company has used Customer Service Suite to save over 10,000 hours of agent time by leveraging chatbots to solve customer queries.
Your chatbot’s human takeover rate may change depending on the role you’re asking it to play. If your chatbot is only handling simple queries, the human takeover rate should be low. If you’re trying to have your chatbot handle more complicated tasks, you may want to be more tolerant of a higher takeover rate while you fine-tune it.
6. Customer satisfaction score
Analyzing your customer satisfaction scores is just as important for your chatbot as it is for your website. If anything, it’s even more important given the role chatbots play in customer service.
Giving users the opportunity to rate your chatbot is one of the best ways to generate a customer satisfaction score. It can be as simple as getting customers to click a thumbs up or thumbs down button after a chatbot interaction or by rating their experience on a scale of one to ten.
You can generate more detailed feedback by asking users to complete a short survey. Questions can be scored on a one to five rating which can be turned into a customer satisfaction score. Take things further by comparing your chatbot customer satisfaction score to your agent’s customer satisfaction scores. If your chatbot is almost as well received as your customer service reps, then you’re clearly doing something right.
7. Retention rate
Your chatbot’s retention rate is the percentage of users who return to speak to it again. It’s a measure of your user’s satisfaction with your chatbot. The higher your retention rate, the better. If customers are willing to use your chatbot again, it’s a good indicator that it’s working effectively.
Improve low retention rates by improving your chatbot’s personalization or assessing the questions customer answers. If you can change missed utterances and human takeovers into chatbot-provided solutions, users will be more willing to use your chatbot again in the future. You can measure this metric over a given period to see if the improvements you’ve made have been effective at increasing retention rates.
