I get this question a lot.
“How does Watson compare to Dialogflow?”
“What do you think of LUIS?”
So I am going to discuss my opinions in this article.
The first disclaimer is that I am clearly biased. I actually work with clients to help them out in their Dialogflow projects. At the same time, I am hoping this post starts a discussion amongst people and helps people do some homework before selecting the appropriate framework.
The second disclaimer is that I haven’t had time to actually work on bots using the other frameworks, and used my knowledge of Dialogflow to consider what I feel are the important features. Then I watched a bunch of online tutorials to check how to implement those same features in the other frameworks. It is definitely possible I missed a thing or two during this process.
Chatfuel and ManyChat
Let us get something out of the way right away.
Chatfuel and ManyChat are not really NLU (Natural Language Understanding) based tools. They have some kind of rudimentary keyword matching, but once you get into features like entities and contexts, they fall quite short.
I have added them to this list mainly to point out this difference to folks because there are a lot of people who say “Chatbots are toys” because they have only ever been exposed to these two platforms.
There are a lot of challenges in building complex multi-step conversations in chatbots. So I am not claiming chatbots are “extremely powerful” tools yet. But they certainly aren’t toys. In fact, some people have become tired of the word chatbot because it covers such a wide spectrum of capabilities and doesn’t even represent anything specific anymore.
For example, a service which allows you to periodically broadcast messages to your Facebook followers may be very useful for you as a marketer, but
- it isn’t really a chatbot even if the user interface is a chat app
- there isn’t really anything “intelligent” about it
- you are confusing everybody when you claim that you “used a chatbot to increase your conversion rate by a gazillion percent!” (instead, simply say you used Chatfuel’s sequences to improve your conversion rate)
So, can a Chatfuel bot do what a Dialogflow powered bot can? No
Is Chatfuel vs Dialogflow a valid or logical comparison? No
Does it mean you should avoid Chatfuel? No
But is Chatfuel a good choice for AI (NLU) bots? No
Are you confused yet? 🙂
What I am saying is, learn both the pros and cons before making your choices.
You should of course use all the tools at your disposal appropriately, but it is also important to learn a bit about what is going on under the hood.
Now back to the main topic.
So here is how I will evaluate the frameworks: I will categorize them based on their ability to provide the building blocks I have already used in Dialogflow.
Here are the major building blocks:
- Explicit Contexts
This is the video I followed for understanding how Amazon Lex works.
This is the video I watched for learning about LUIS and the bot framework.
Here is the playlist I watched to understand how IBM Watson Assistant works.
I believe all the frameworks are more or less equally capable in this particular feature.
All of them provide (somewhat) easy ways to define intents, as well as use multiple training phrases to make the intent recognition accuracy better.
While different frameworks may differ in their power when it comes to extracting entities, all of them have a concept of a developer entity and a system entity.
Here again, they are quite similar to each other.
Explicit contexts which have a lifespan
Contexts are used to keep track of what has happened in the conversation till the current moment. I think this is where the difference between the frameworks becomes quite stark.
I was looking for a word to express how contexts in Dialogflow are different from those in the other frameworks, and I came across this article.  It is a very good article, quite well researched, and defines the notion of “explicit contexts”. Explicit contexts are defined right inside the visual editor (so you can see their names) . Also, explicit contexts can be used to guide the conversation flow so you can follow the conversation logic – said another way, you can start from a flowchart and use the help of explicit contexts to quickly turn it into a working chatbot.
You are probably familiar with how contexts work in Dialogflow (if not, read this article)
How do they work in the other frameworks?
- Lex has no contexts, but it has session variables which are used to store the state of the conversation, including user inputs from previous messages. (This isn’t from the video. In fact I couldn’t find any mention of contexts in Lex tutorials, and then learnt that they use the concept of session variables to save conversation state).
- Lex considers slot filling as the fundamental operation of chatbots. Unfortunately IMO, the other bot frameworks (including Dialogflow) seem to have copied this bad idea. Thankfully, in Dialogflow, there are ways around it.
LUIS doesn’t have a visual editor (that I am aware of). Microsoft seems to have skipped no-code and low-code and have created a only-code tool. Contexts are thus declared and used from code From my understanding, there is no notion of context lifespan
- This isn’t entirely true now, they have created a visual dialog builder (see comment below the article). There are still no explicit contexts in LUIS, which is not only useful for constructing conversation flows, but also important for narrowing down the candidate intent list. So I still don’t rate the LUIS very highly but if you are doing all your work in the MSFT stack, it is a much better option now.
- Watson Assistant uses a sort of linear ordering of the intents to provide priority to the different intents. This is definitely unique and different from the other frameworks
- Watson does have the notion of contexts, but it doesn’t seem to have any way to specify the lifetime (lifespan) of a context.
- It doesn’t seem like there is some way to actually use contexts themselves to guide the conversation (see for example my Dialogflow based decision tree bot)
In other words, the combination of explicitness of contexts and the ability to specify a context lifespan provides the bot builder with a lot more options in Dialogflow.
All the frameworks support the notion of webhooks, so I am not sure if there is much to choose between them.
What about RASA NLU?
RASA NLU is hard to compare to the list above, because
a) it is open source, and
b) theoretically, it can be modified to do whatever you want. This isn’t true of the other frameworks which are essentially running on other companies’ servers.
However, I mention in my article comparing RASA and Dialogflow that I think RASA NLU really missed a trick by not evolving their framework to allow for explicit contexts and wildcard entities.
Having said that, one of the huge benefits of RASA is that it is really flexible and you can sort of mold it to your requirements. But the lack of the features I have mentioned means that you need someone who has a solid background in Natural Language Processing (NLP) to actually utilize RASA to its fullest potential.
Dialogflow effectively reduces your risk by
a) providing a solid framework backed by one of the world’s largest companies (although I do wish their Dialogflow support was better)
b) allowing you to hire programmers who don’t have a background in NLP. It is fairly easy to prove to yourself that there are more programmers who don’t know NLP than those who do know NLP 🙂
I have written an eBook to explain these points as well as my views on Dialogflow vs RASA.
What do you think? Have you already taken out your pitch-fork in defense of your favorite framework? 🙂
Let me know why I am wrong. I found a similar article, also a very good one, which compares these frameworks here.
- Dialogflow CX vs ES: First look
- Seven ways to integrate a Dialogflow chatbot into your website
- Dialogflow Zobot: Selection Triggers the next intent
- Dialogflow Architecture
- Dialogflow Python webhook tutorial
- Dialogflow training
- Dialogflow vs Lex vs LUIS vs Watson vs Chatfuel
- Convert your WordPress website into a Dialogflow FAQ chatbot
- When NOT to use follow up intents in DialogFlow
- Dialogflow Context Lifespan