First published: Jan 2018 | Last update: Oct 2020
I was recently asked by a client if it is possible to bulk upload intents from a CSV file to Dialogflow.
Apparently, there was a time when this feature was already available in Dialogflow. But it isn’t straight forward anymore, and its probably for a good reason.
CSV file structure
Typically, the CSV file structure is simple. There is a column with a user says message, and then another column with the text response. What you want to do: You upload this CSV format, and you have an agent at the end of it.
Things to consider
Now, this simple format is not sufficient. For example,
- how can you handle multiple user says messages?
- what if you want to add multiple text responses?
- what should be used as the intent’s name?
An alternative file structure
Suppose we create the following file structure to accommodate the points from the previous section. You have an intent ID, followed by intent name, followed by userSays phrase, and the final column will be your text response.
While this gives you the ability to use multiple userSays phrases as well as multiple text responses, it is also more complex to process this format. But there is still a major issue when it comes to using this: the follow up intent.
Sometime around May 2017, Dialogflow (which was then API.AI) introduced the followup intents feature. These allowed you to have a hierarchy of intents, like below.
Since the number of hierarchical levels is (theoretically) unlimited, you can imagine that it is not possible to capture this very complex structure in the CSV format at all.
Doing bulk upload using Import/Export functions
Does this mean there isn’t a way to use the CSV file and upload your intents directly? Actually you still have another option.
You can do it using the Import/Export ZIP files function.
Here is the summary:
- Prepare a set of JSON files representing the CSV files you wish to upload
- Make sure the JSON files are created using a specific file and directory structure
- ZIP the top level directory
- Import or Restore from the ZIP file you created
First, let us export a ZIP file for a simple project and take a look at the directory and file structure, as well as the contents of the JSON file.
I will use an agent which has exactly two intents (taken from the prebuilt smalltalk agent).
I went to the Export and Import tab, and clicked on Export the ZIP file for this simple agent. Once you uncompress the ZIP file, the following file/folder structure is revealed:
And you can inspect the smalltalk.agent.be_clever_usersays_en.json.
You see that it is an array of JSON objects, and an individual JSON object has an id, the data, isTemplate, count and updated fields.
Next, notice the response JSON object smalltalk.agent.be_clever.json
Say, you want to add your own intent.
You simply need to follow the same pattern as the other intents, and create two new JSON files, and ZIP it up and import it into Dialogflow.
So you generate a random GUID, and get all the other stuff exactly in place. In my example, I made a copy of both the smalltalk.agent.be_clever_usersays_en.json and the smalltalk.agent.be_clever.json files and added a random GUID in the copied files, and created a ZIP file and tried to do a restore.
Make sure the filenames match:
Here is what the end result looks like after the import:
In fact, an even easier option is to simply omit all the GUIDs in the intents as well as the userSays as you create your ZIP file. Dialogflow will simply see that the ID is missing and generate its own.
But there are a couple of important things to note:
- Dialogflow will not permit two intents with the exact same name – be sure to change the name
- Your files should be of the format filename.json and filename_usersays_en.json – if you don’t follow this convention, the import doesn’t work correctly
Now, how do we use this knowledge to import a CSV file?
If you have been following along, you need to do the following:
- From the CSV file, for each intent, you should automatically generate a JSON file with the appropriate text for userSays and text responses.
- Make sure you structure the JSON files into the folder correctly
- Create a ZIP file from the folder
- Import or Restore the ZIP file
You can write scripts to do this.
I have created a demo app which shows how this works.
1 Choose the bot’s timezone
The timezone is important because it will affect the UTC time for any date time information which is parsed by the bot.
2 Choose the bot’s language
You can choose any language supported by Dialogflow.
3 Choose the CSV format
Right now, the app supports one of three different formats. The basic 4 column format supports only training phrases and responses. The more advanced 8 column format also supports input and output context, output context lifespan, and event. The 10 column format expands on the 8 column format and provides support for a second text response block, as well as a boolean Yes/No value for whether or not the intent calls a webhook.
Explanation of 4 Column CSV format
Sample 4 column CSV file
Sample 8 column CSV file
Sample 10 column CSV file
4 Select the CSV file from your file system
Use the Browse.. button and select an appropriate CSV file from your local computer.
5 Click on Download agent ZIP file
As soon as you select the CSV file, you should see a button appear which allows you to Download the agent ZIP file. Click on the button. You should now see that the agent ZIP file gets downloaded to your local computer.
6 Use the Restore ZIP file functionality in Dialogflow
Go to Dialogflow’s console, and use the Restore ZIP feature to upload the agent ZIP file generated by my app. If the Restore completes successfully, you should see all the stuff from your CSV file appear as intents inside your Dialogflow agent.
- Thanksgiving 2020 Deals
- How to debug Dialogflow Python webhook using ngrok
- EU VAT MOSS Tips for online course creators
- How to integrate Dialogflow into your Flutter app
- How to bulk upload intents from a CSV file in Dialogflow
- Get your DialogFlow agent to initiate the conversation before user types a message
- Why doesn’t Dialogflow CX support more 1-click integrations?
- Dialogflow CX vs ES: First look
- Dialogflow Quickstart Templates
- Dialogflow Zobot: Selection Triggers the next intent