I have been asked many times if Dialogflow can automatically handle typos. The short answer is that it cannot.
So how can we handle typos from our users? Here again, there are two types of typos* - typos in entity values and typos in regular words.
I am not sure if there is any sensible way to handle typos in regular words, because Dialogflow's pattern matching is quite broad. In other words, you cannot really anticipate all the possible variants that the user could type.
On the other hand, typos in entity values are both more likely and in some ways easier to handle.
Here are a bunch of suggestions for handling typos in entity values. What you ultimately choose depends on how accurate you want your typo correction to be.
The official recommendation
First, let us look at the official recommendation from the Dialogflow team:
Option 1: Add synonyms with common mis-spellings into the entity value
In this case, you simply add a common mis-spelling into the entity value in Dialogflow:
If you read the thread I linked to in the previous section, obviously the Dialogflow folks don't recommend this option. However, this is easily the most elegant solution. Why? Because you are letting Dialogflow do all the heavy lifting.
When that happens, everything else becomes much easier -
- you are letting Dialogflow use its secret sauce of intent mapping + entity recognition so the NLP stuff has been completely handed off to the NLP engine
- you get greater consistency in intent mapping (versus if you were to intercept it in some way, for example by rewriting the user query)
- your conversation flows along expected lines, which helps out your analytics later
However, it may not always be feasible to do it. If the number of entity values you have in your chatbot runs into the hundreds and thousands, you probably have no way to add these hand-coded changes to your entity values.
Option 2: Use an online spell-checking API like Bing
Microsoft Bing has an online spell checker API which is considered to be quite good (and a past client mentioned he found it pretty useful to fix typos in Dialogflow user phrases too). Note that it is a paid service, but it is also quite nominally priced.
Here is the JSON returned by the service (note that it contains an offset explaining what was misspelled):
Naturally, since this is Microsoft we are talking about, you don't get the actual corrected query all in one piece in your JSON until you jump through a few hoops.
Option 3: Roll your own custom code
With an online spell check API such as Bing, one problem is with entity names it doesn't know about. For example, if it already knew my name, it would split the phrase on the left accordingly into first and last names.
In this situation you are looking at adding your own code to handle such user queries. You would need to first split the sentence into words, and then lookup a custom dictionary (generated based on known entity values) and then try and do your own spell correction.
Needless to say, this is not something that will be trivial.
Some tips to make it work better
You are likely to select option 2 (use an online spell check API) if this is an important problem for you. You can follow these tips in that situation:
Only query the spell check API when you hit the Fallback intent
In other words, your service will look at the result from Dialogflow's intent mapping, and choose to query the online spell check API only if Dialogflow cannot map to an intent. If it fails, you have two options,
- you can either pass on the failure message to the user and give them an easy way to re-submit with the corrected spelling (ala Google's "Did you mean?")
- you can do it automatically on the user's behalf and return the second result back to the user
I prefer the first option because it is more transparent to the user and in some corner case, your spelling correction itself might be incorrect and the user gets a chance to override your change.
If your UI supports it, turn the second attempt into a selection box
I have written about this in another article, where I called this the FallbackToList pattern, but if your UI supports something like this, you can actually make the second submission a selection rather than asking the user to type. This only works when you have actually narrowed down the potential corrections to a handful, as you might expect. Also make the last option in your selection box "No, just use my original message" to let the user override your change.
*I challenge you to say "two types of typos" 10 times in a row - I bet that's a tongue twister 🙂