Analytics | DialogFlow

Dialogflow Conversation Analytics Tips

The ability to do conversation analytics in Dialogflow is directly tied to the accuracy of your bot’s intent mapping. (Here is an article which talks about measuring your Dialogflow bot’s accuracy) This is why Chatbase is not a particularly useful tool for Dialogflow conversation analytics. Unlike Google Analytics funnels, where you can be certain that a user click is exactly what shows up in the “funnel”, the same thing cannot be said about Chatbase funnels (read my article on Chatbase funnels if you haven’t used them before) which actually rely on Dialogflow’s intent mapping being accurate. Even in their documentation, Chatbase recommends that you manually inspect the transcript to ensure that the correct intent was mapped. They recommend that you…

Analytics | DialogFlow

How do you measure your Dialogflow bot’s accuracy

I am working on creating a new service – tentatively called Dialogflow conversation audit – where I plan to analyze the client’s agent and make suggestions to improve its accuracy. But this means there must be a measurable way to determine the bot’s accuracy in the first place. The U-M-M method A while back, I wrote a post which linked to Chatbase’s UMM method which provides a way to reason about your chatbot’s accuracy. While it is a good idea and I do derive some ideas from it, it is not particularly useful because there isn’t any way to measure the accuracy using the UMM method. What I am proposing here is much simpler[1]. The confusion matrix You may be…

Analytics | DialogFlow

How to use Chatbase to improve your bot

I recently spent some time watching the Chatbase team’s video on improving chatbots using Chatbase. I will summarize the material in this post. What is UMM? UMM is an acronym which stands for: U – Unsupported requests M – Misunderstood requests M – Missed requests Let us take a look at some examples to show what each of these types of requests are. Intent Mapping For the sake of explanation, let us suppose you can assign a color to each of your intents. So you have red, blue and green intents, and the black color represents the Fallback intent. Unsupported requests Suppose, we get a new user request and it is an orange. We don’t (yet) have any way to…