In this tutorial, you will learn the building blocks of Dialogflow by developing a chatbot which can answer user questions about a specific topic (Planets) by fetching the replies from a database.
This guide is meant for technical non-programmers. At the same time, you will still be able to get some useful stuff from this guide if you are a programmer new to Dialogflow.
A small note before you start
Even though Dialogflow is quite friendly for non-programmers, you still need to be fairly technical to be able to use it (this is why I repeatedly use the phrase "technical non-programmers" on my website).
In that sense, this tutorial is also a good test for your technical skills.
What we will be building
We will be building a PlanetsBot. What can the PlanetsBot do?
It can answer questions about planets, naturally!
Notice that in the first response, the bot is able to understand that we are asking for a specific attribute (volume) of a planet (Mars). And then in the second response, it understands that "its" refers to the planet Mars.
The answer that you see (e.g. "163 billion cubic kms") is coming from a database. In other words, it is dynamic, and you can change the answer by updating the data in the database.
Build the basic bot without the webhook
We will first learn the basic concepts of intents, entities and contexts before building the webhook. These are foundational concepts that you must know if you want to build non-trivial Dialogflow bots.
Intents
First, we will take a look at intents. Intents define what the user wants - i.e. they represent the user's intention.
Entities
Dialogflow entities allow us to capture values for specific types of objects.
Contexts
Contexts help you keep track of what has already happened in the conversation.
Note: without doing the step above (that is creating the basic bot without the webhook) the rest of this tutorial will not be of any help.
Before you build your webhook
Now, we are ready to build the webhook which will actually fetch and display the data inside our PlanetsBot.
There are just a couple more things to do:
- Set an action for each intent which calls the webhook
- Make sure you turn on the "Call webhook" toggle on each intent
Here is what the UserAsksForAttributeOfPlanet intent looks like:
Note that the action is called getPlanetAttribute and we have toggled the "Enable webhook call for this intent" switch.
Similarly, this is what the UserChangesAttribute intent looks like:
The third intent is homework for the student. 🙂
Action
The action is a way to tell the webhook what piece of code you wish to execute.
For example, in the actual code which fetches data from the database, there are two code blocks.
The Action that you set will determine which code block is used.
Note, the code for these code blocks are actually from the RunKit notebook which I talk about later in this tutorial.
The best database for non-programmers building Dialogflow bots
For non-programmers, working with spreadsheets is easy. But working with databases is not. A good compromise is Airtable - which is a sort of a combination of a spreadsheet and a database.
On the one hand, it is trivial to input data into Airtable because everything looks like a grid of cells.
On the other hand, you can do some database like operations - notably enforce data types and create links between table records - that you cannot do in regular spreadsheets like Excel and Google Sheets.
Finally, Airtable provides an easy to use API to move data in and out of their tables (if you ever need to migrate away) and also has probably the best UX for non-programmers who wish to do complex workflows without learning how to code.
All this to say, I have decided to use Airtable as the database for this tutorial. 🙂
Note: while I call it the best database, in reality, actual databases are much, much more powerful than Airtable. However, installing and maintaining and getting the most out of such databases is not easy (even for programmers).
Airtable Basic Concepts
Instead of trying to define them, I will just link to this Wikipedia article which explains it succinctly.
Create an Airtable table for the Planets data
You don't have to input the data by yourself. You can simply use the data from my Airtable table.
1 First, sign up for Airtable (referral link - I get free Airtable credits if you sign up and confirm your account).
2 Make sure you are logged in to your Airtable account.
3 Then click on the Copy Base link at the bottom of this table and you will have a replica of all the data inside your own Airtable account!
Build the webhook
Once you have your Airtable "database" ready, it is now time to write the webhook code.
However, you will not be writing any code, because it is possible to "clone" my Runkit notebook and get the code for your webhook.
Here are the steps:
1 Create a Runkit account
First go here and create a Runkit account.
2 Clone the Runkit notebook
Then, go to the URL where I have created my Runkit notebook. You should be logged in to Runkit to be able to do this step correctly.
Then click on the Clone and Edit this document at the top of the page.
3 Click on endpoint link
Click on the endpoint link at the top of the document.
You will actually see the following error: (don't worry, it is expected)
4 Get Airtable API key
Log in to your Airtable account and go to Account settings.
Copy your Airtable API key and save it.
5 Get Airtable Base ID
Now go to the API page of your Airtable account. Again, you should be logged in.
Click on the Planets base you just copied into your Account. (Note: you probably will not see as many bases in your account if you just created it).
Click on the Authentication link on the left panel.
Now select the JavaScript tab at the top, and copy the baseID as shown below
6 Add API Key and Base ID to Runkit environment settings
In your Runkit account, go to settings. (Again, you should be logged in)
Scroll down to the Environment Variables section.
Now add an environment variable called AIRTABLE_API_KEY and paste the value you copied in step 4. Then create another environment variable called PLANETS_BASE and paste the value you copied in step 5. Runkit automatically saves these values once you paste them in, so no need to explicitly save them.
7 Test the endpoint
Now go to the AirtableDF notebook you cloned in step 2 and click on the endpoint link again. This time you should see some actual data coming back from your Airtable table.
As a sidebar, sometimes when you access this URL, you will get an error message like this instead:
While I don’t know enough about RunKit’s internal architecture to explain what is going on here, the error seems to be a temporary one and goes away in a couple of minutes.
Even after those two minutes, if you don't see some output message such as "The Volume of Earth....", your set up is not working properly. Retrace your steps.
8 Publish the notebook
Click on the publish link.
You don't need to fill anything out. Click on the Publish button in the popup.
9 Copy the endpoint link
Now click on the endpoint link again after publish.
Copy the endpoint URL from the browser bar. Note that it ends with .sh.
10 Paste the URL into Dialogflow fulfillment
Go to your Dialogflow agent and paste the URL into Dialogflow fulfillment.
Don't forget to save it by clicking on the blue Save button. 🙂
11 Test it in your Dialogflow simulator
Try a query like "what is the color of Mars" inside the Dialogflow simulator. You should see a response similar to the following.
Not working?
Access the endpoint in your browser, just as you did in step 7. Do you see an error message? If yes, wait for a few minutes and try again.
On the other hand, if you do see a proper output when you access the endpoint in your browser and you see your query fail inside the simulator, there is something else going on. Please copy the error message from inside the DIAGNOSTIC INFO of your Dialogflow simulator and leave a comment with the error message that you see.
12 Update the data inside Airtable and test again
Now, you actually have an end to end webhook working. Not only that, it is actually fetching data live from your Airtable table.
To test this, tweak the value of the attribute above.
Test it in the Dialogflow simulator again. All data changes happen in real time in Airtable, so you should see the updated value right away inside the simulator.
Summary
If you have reached this far and have gotten everything working without any issues, congrats! You have built a working webhook which accesses data from a real database using Dialogflow.
Now you can fill out all the entries marked as n/a in the Airtable base with real data and complete the PlanetsBots. 🙂
Ignore my previous comment. I fixed the issue. Thanks.
How was it fixed coz I am getting the same error nine days after you post this
I see a proper output when I access the endpoint in the browser but query fail inside the simulator with default response: “This is an endpoint published to RunKit”
Its only case sensitive.. How can i make case insensitive ? Eg: It only accepting “Earth” not “EARTH”.
Thank You in advance
The simplest option in this case is to add the all-uppercase value EARTH as a synonym into the appropriate row in the @planets entity.See answer belowI just checked this, and confirmed that usually Dialogflow’s entity extraction is NOT case sensitive. For example, it is able to understand EARTH even if I only have earth (lower case) in my entity value. I suggest going through the tutorial more carefully, you probably missed a step.
I double checked, it was working perfect if im using exact names in the coloum or row in airtable. But the only error im getting is if im checking a value from webhook (in this tutor its airtable) A1 (Earth) has to be exact like
Earth (not EARTH or eartH or like that) then its getting a false answer or no answer. Please sir i can upload a video also if you wish. Airtable value is very case sensitive in this method i see.
Thank You
Make sure that the Dialogflow entity key (the value in the first column of an entity) exactly matches Airtable. And remember that Dialogflow provides the normalized value (the key) so you shouldn’t be facing any problem with Airtable lookup.
Can i able to perform calculations from the data and display according to the input
.
Yes, you can do that from your webhook code.
Thank you for making this tutorial.
Can you tell me what I have to do:
Webhook call failed. Error: DEADLINE_EXCEEDED.
That’s a tough one to debug, as it can be caused by different things. Basically, your webhook call is not responding within the time allotted (5 seconds). My suggestion is to go through the tutorial again slowly and see if you missed something.
Hmm I did it and created the agent and so exactly like you (with your files/changed the webhook links) and its the same failure.
Maybe you can propose another database which responds quicker?
At the moment, I don’t have any simple answer for your question. The way this tutorial was created, it was based on being able to access Airtable via the webhook because Airtable makes things easier for non-programmers.
If you need very urgent help, you can book a 1-on-1 paid video consultation.