Contentful BlogIn 2019, I was boarding a plane for London when I realized that I forgot my shaving kit. I quickly opened the app for the hotel that I would be staying in. Thanks to its chatbot feature, I dropped them a request. They were able to immediately solve my shaving problem and, in addition, booked me a taxi from the airport.
The benefits of chatbots are pretty evident: they make customers happy, they operate as a sales tool and allow you to maintain a conversational relationship. I didn’t get frustrated, I stayed connected with the hotel and now we’ve built a conversational history.
Because conversational bots offer an important channel to reach customers, most brands are developing expert bots that provide specific solutions. Some chatbots provide FAQ solutions, others recommend specific products depending on the customer’s needs, and still others quickly reply to questions about billing and account information. Contentful can handle the content layer of your chatbot, and this post walks you through the architecture, implementation and results.
Specific context and tech stack for chatbot
One of our enterprise customers wanted to build an FAQ chatbot with Contentful after they used the content platform to successfully power their web products. They were already using Microsoft cloud services, so we decided to build a quick proof-of-concept using Microsoft Azure functions and knowledge base. Contentful works with these tools, but can also be used with other cloud and bot services like Amazon Web Services, Google Cloud Platform, or IBM Watson thanks to its APIs. After some initial investigation, we developed the architecture presented here, which uses Contentful webhooks, Azure cloud functions and Microsoft’s knowledge base service.
Architecture
Publishing flow
Editors will publish the FAQ in Contentful
Publishing of the FAQ will trigger a webhook in Contentful
The webhook will call our publish cloud function
The function will push the content to our knowledge base
Unpublishing flow
Editors will unpublish the FAQ in Contentful
Unpublishing of the FAQ will trigger a webhook in Contentful
The webhook will call our unpublish cloud function
The function will remove the FAQ from our knowledge base
Implementation
Contentful
We used Contentful to store questions and answers for the chatbot. In the Contentful web app, editors can easily write, edit and publish that content.
Content Model
We’re keeping the content model very simple for this demo — just a single content type holds our question and answer pairs, as well as an internal title for content editors to have as a reference.
Webhooks
We use webhooks for publish and unpublish events, triggering our cloud functions.
When creating the publish event webhook, make sure to set the trigger of the webhook for the entry publish event as shown below. For the URL, we will need to add the endpoint of our cloud function, as we have not configured any cloud function yet. We can use the DummyWebhook website or ngrok to create dummy endpoints and add them.
When creating the unpublish event webhook, make sure to set the trigger of our webhook for the entry unpublish event as shown below. Just like last time, we’ll populate the URL with a dummy value from DummyWebhook or ngrok.
Once done, you can create an entry and test our workflow by publishing and unpublishing it and checking the webhook payload being sent. You can then view the payloads in our webhook logs.
Publish entry webhook payload:
{
"sys": {
"type": "Entry",
"id": "1ssZF4Lqq1Wpi9KSHvq9jE",
"space": {
"sys": {
"type": "Link",
"linkType": "Space",
"id": "bwn0phmhnub6"
}
},
"environment": {
"sys": {
"id": "master",
"type": "Link",
"linkType": "Environment"
}
},
"contentType": {
"sys": {
"type": "Link",
"linkType": "ContentType",
"id": "faqQuestion"
}
},
"revision": 2,
"createdAt": "2020-04-13T23:06:07.225Z",
"updatedAt": "2020-04-13T23:07:05.635Z"
},
"fields": {
"title": {
"en-US": "First question"
},
"question": {
"en-US": "What is contentful? \n\n"
},
"answer": {
"en-US": {
"data": {},
"content": [
{
"data": {},
"content": [
{
"data": {},
"marks": [],
"value": "Contentful is Headless CSM",
"nodeType": "text"
}
],
"nodeType": "paragraph"
}
],
"nodeType": "document"
}
}
}
}
Unpublish entry webhook payload:
{
"sys": {
"type": "DeletedEntry",
"id": "1ssZF4Lqq1Wpi9KSHvq9jE",
"space": {
"sys": {
"type": "Link",
"linkType": "Space",
"id": "bwn0phmhnub6"
}
},
"environment": {
"sys": {
"id": "master",
"type": "Link",
"linkType": "Environment"
}
},
"contentType": {
"sys": {
"type": "Link",
"linkType": "ContentType",
"id": "faqQuestion"
}
},
"revision": 2,
"createdAt": "2020-04-13T23:27:30.971Z",
"updatedAt": "2020-04-13T23:27:30.971Z",
"deletedAt": "2020-04-13T23:27:30.971Z"
}
}
For now, that’s all we have to do within Contentful.
Creating the FAQ chatbot in Microsoft Knowledge Base
QnA service
1. Go to https://www.qnamaker.ai/ and create a free account.
2. Create a knowledge base.
3. Create a QnA service.
4. Go back to https://www.qnamaker.ai/ and refresh to see your subscription name and Azure QnA service.
5. Complete the rest of the steps to create a knowledge base. Once done, you will have a knowledge base similar to the one shown below.
APIs to integrate
Now that we’re done with creating our knowledge base, we need to populate it with questions and answers coming from Contentful for the chatbot to use. Let’s explore which APIs we can use to add, update or delete information from our knowledge base.
We will be using the update endpoint to add and remove content. We will also use the download endpoint to get all questions and answers and then filter them.
Azure functions
Azure functions are a powerful way to develop and deploy serverless applications. For this use case we chose Azure functions as we can quickly develop and test our proof of concept.
Setup
This video tutorial from Microsoft gives you a guide to set up an Azure function project using Visual Studio Code.
We used the following settings for this chatbot project.
Language: you can select multiple languages, but we used JavaScript
Function trigger: http
Name: the function we used publishWebhookHandler for the first function and unpublishWebhookHandler for the second function
Access Rights: set it to anonymous for now. For production, make sure to implement proper security mechanisms
We created a new Azure function project with two functions: publishWebhookHandler and unpublishWebhookHandler.
publishWebhookHandler
This function will be used to push content into our knowledge base using the webhook coming from Contentful whenever the content is published.
Algorithm
Retrieve the following from the webhook: entry ID, question and answer
Make an API call to our knowledge base to get all the questions and the answers. Once you have all questions, filter the question using metadata and check for the entry ID. We’re doing this to check if we have duplicate questions in our knowledge base, so we update them instead of adding a duplicate entry.
If the question is present we will update that question using the update endpoint. Use the same endpoint to add new questions. The only difference is the body of the payload. You can see in the metadata that we are adding key contentfulid and its value. This helps us to map entries in Contentful with entries in the knowledge base and will help us for deleting and updating entries.
Adding a new question:
{
"add": {
"qnaList": [
{
"answer": "answer",
"source": "source",
"questions": [
"question"
],
"metadata": [{"name":"contentfulid","value":"1ssZF4Lqq1Wpi9KSHvq9jE"}]
}
]
}
}
Updating a question:
{
"update": {
"name": "QnA Maker FAQ Prompts Bot",
"qnaList": [
{
"id":87,
"answer": "answer 2",
"source": "source",
"questions": {
"add":["questions 1"],
"delete":[]
}
}
]
}
}
Sample function for adding content to our knowledge base:
module.exports = async function(context, req) {
if (req.body) {
const contentfulID = req.body.sys.id;
const contentfulQuestion = req.body.fields.question['en-US'];
const contentfulAnswer = req.body.fields.answer['en-US'].content[0].content[0].value;
const questionFromKB = await getQuestionIdFromKB(contentfulID);
const response = await upsertQuestionInKB(context, questionFromKB, contentfulID, contentfulAnswer, contentfulQuestion);
context.res = {
status: 200,
body: JSON.stringify(response),
};
} else {
context.res = {
status: 400,
body: 'Please pass a name on the query string or in the request body',
};
}
};
unpublishWebhookHandler
This function removes content from our knowledge base using the unpublish webhook, whenever content is unpublished.
module.exports = async function(context, req) {
if (req.body) {
const contentfulID = req.body.sys.id;
const questionId = await getQuestionIdFromKB(contentfulID);
const response = await removeQuestionFromKB(questionId);
context.res = {
body: JSON.stringify(response),
};
} else {
context.res = {
status: 400,
body: 'Please pass a name on the query string or in the request body',
};
}
};
Algorithm
Retrieve the entry id from the webhook
Make an API call to our knowledge base to get all the questions, and filter the question using the entry id
Make an API call to remove the question from our knowledge base. The payload will look similar to the following:
{
"delete":{
"ids":[
29
]
}
}
Deploy
Once both functions are ready, they can be deployed to Azure Cloud.
When the deployment is complete, get the publishWebhookHandler and unpublishWebhookHandler URLs from Microsoft Azure portal. Add them to the webhooks, which we created in the first step.
Testing our chatbot end to end
Populate content
We published three Q&As for testing in Contentful. Each of the publish events triggered a webhook and they were pushed into our knowledge base. To check, we logged into https://www.qnamaker.ai to see if the questions are published.
On the screenshot below you can see the three published Q&As on Contentful as well as added on our knowledge base.
Now, our knowledge base will train the content. Inside QnA Service, clicking on Test you can test your chatbot.
Results
Now you have an intelligent chatbot customers can talk to. This way customers do not need to go through the extensive list on a website searching for a simple answer. You can also measure the sentiments of the customers and their frequent questions. With this additional information, we are empowered to improve our FAQs.
Future enhancements for our chatbot
Our content model can be extended to add multiple questions for a single answer, and for every entry, additional metadata could be added. For this we have to update the sync functionality within the publishWebhookHandler function.
Contentful also provides different localization patterns, which can be used to create content in multiple languages, allowing us to power multilingual bots.
In this article, we explored using the Microsoft tech stack, but similar things can be achieved using AWS Lambda and AWS Lex. Any chatbot platform that provides API access to their knowledge base can be used with Contentful. Using webhooks and cloud functions allow us to push content into a knowledge base and train it afterward.
If you’re looking to implement your own conversational chatbot, you can sign up and try it for yourself. Feel free to contact us. Our solution services team works with enterprise customers to accelerate their digital journey.
