If your website is used to onboarding and answering your customers' questions, you’re likely familiar with chat plug-ins such as Olark. Olark allows you to easily install their chat app right there on your web page and provides a direct line of communication with your customer.
You will find that your customers commonly ask some of the same questions. This ends up tying up your chat operators’ precious time that could be better spent taking care of particular, ad-hoc queries. And here is where F.A.Q (Frequently Asked Questions) pages on your site make all the difference for your team’s productivity. FAQ sections are commonplace for many businesses with online presence.
An FAQ section is meant to be composed of a set of frequently received questions and their associated answers. You can check out ButterCMS’ FAQ information here.
When it comes to managing these questions, using ButterCMS’ Write API is the perfect solution because it simplifies the work required to add and maintain these FAQs on your site.
Now, here is where the problem lies: understanding which questions commonly come up from your customers that would be best suited for your FAQ page. Typically you’d need to comb through past chat logs and manually decide whether or not to add them. Instead, with ButterCMS’ write API you can directly send the relevant chats to your ButterCMS account when a particular chat contains an FAQ. Wouldn't that simplify your task of keeping up with your customer’s growing need for answers?
I’d say so, and in this article, I’m going to show you how to add just enough code to hook Olark and ButterCMS’ brand new Write API in order to achieve this exact thing.
The Solution
Because this is mostly a chat-based interaction, the logic trigger that’ll spark all the automation will happen over at Olark’s WebHooks. Instead of having to set up something over at our ButterCMS’ admin console, we’ll only need to create an FAQ page type over there (more on that in a second).
But for our chat service, we’ll have to set up a particular WebHook that’ll send the information to our code. With Olark’s webhooks we have two options. We either:
- Ask the chat operator to manually trigger the webhook (yes, hook as in singular, you can only set one up ).
- Have our hook get triggered every time a conversation is closed (either manually using the
!closecommand or by inactivity)
In either case, only one hook is available. This should not be a problem, because you can tag your customer conversations in Olark with whatever tagging system you develop. You can then decide what to do with the received hook payload based on things like those chat tags.
With that being said, let me quickly go over the steps for our automated solution:
- Upon identifying a currently open chat as containing an FAQ, the operator will tag the chat with the “faq” tag (using the command “
!tag faq” on Olark’s chat UI). - Once the answer is provided, they will use the “
!webhook” command. This will manually trigger a webhook sending the current conversation to one of our scripts. - That script will make sure the FAQ tag is present and it’ll create a new page with its raw content in ButterCMS. The page will be of type “FAQ” and will be unpublished by default.
The following diagram represents the flow described above:
The final output of the above flow is a new page of page type “faq” created in your ButterCMS account containing the full conversation with the user.
It would be quite difficult to automatically extract the question and the actual answer from the transcription of the conversation therefore simply notifying your CMS admins that there is a new FAQ to be reviewed should set your team up for success.
Setting things up in Olark
Note: Using automation features such as WebHooks are not part of the free offering Olark provides. You can easily test them for free if you keep your tests to under two weeks, which is when the trial period ends.
Setting up your webhook
The following screenshot shows where Webhooks can be found: inside the Integrations > Automation section of Olark.
Once in there, you’ll get the following dialog box to set up your hook:
Let me quickly review the options for you:
- URL to post to. This is the URL where your script will be running. Just like with ButterCMS, the hook sends a POST request to a particular URL, so we set that up here.
- Send all transcripts automatically. The hook sends conversation transcripts (I’ll show you their structure in a moment) every time your chat operator uses the
!webhookcommand from within the chat UI. If you check this box, the transcript will be sent automatically once the conversation closes (either by using the!closecommand or by inactivity from the user). - Send offline messages. This one is pretty straightforward. Any message received while your operator is offline will not be part of the transcript unless this box is checked.
- Send all events. This is used to obtain additional details as to what is happening with the conversation. If this box is checked, the transcript will be sent on every event. That being said, their documentation doesn’t really list all events (only the “Start Conversation” event).
- Disable. This simply disables the hook.
For our particular needs, we can get away with just setting up the URL and then hitting “Save".
And with that, we can move on to actually writing some code. Although not to worry, it’s just a little bit of Node.js that you'll need.
The Hook’s code
Because we need to have support for a POST request, we’ll simply go with an Express.js API with a single endpoint (POST /olark-hook).
Once a POST is received, our code will:
- Get the actual payload sent by Olark
- Check to make sure the tags list contain our "faq" tag
- If so, create a new FAQ page in ButterCMS
Our FAQ pages will have three fields:
- Question. With the actual question being answered.
- Answer. With the actual answer to the question
- Transcript. With the full transcript of the conversation. This will be used by a human operator to populate the above two fields.
In other words, our brand new page created through code will have its first two fields empty. And to simplify the operator’s life, we’ll clean up the transcript just a bit.
Payload structure
Olark’s JSON payload will have the following structure:
{
kind: 'Conversation',
tags: [ 'faq' ],
items: [
{
body: 'hey there!',
timestamp: '1561209998.139551',
kind: 'MessageToVisitor',
nickname: 'Fernando',
operatorId: '1046479'
},
{
body: "I'm wondering where do you live?",
timestamp: '1561212360.810266',
kind: 'MessageToOperator',
nickname: 'Uruguay (Montevideo) #2809',
visitor_nickname: 'Uruguay (Montevideo) #2809'
},
],
operators: {
'1046479': {
Username: ‘operator-username’,
emailAddress: op@gmail.com',
kind: 'Operator',
nickname: 'Fernando',
id: '1046479'
}
},
visitor: {
city: 'Montevideo',
kind: 'Visitor',
conversationBeginPage: 'http://localhost:4000/',
countryCode: 'UY',
ip: '186.50.56.130',
chat_feedback: {},
operatingSystem: 'Linux',
emailAddress: '',
country: 'Uruguay',
organization: 'Administracion Nacional de Telecomunicaciones',
fullName: 'Uruguay (Montevideo) #2809',
id: 'V5i8QqkA8fVEIhE68O7Lr0S8BbO65Ir2',
browser: 'Chrome 75.0.3770.80'
},
id: '2Ts7DBJRm1rZrKt58O7Lr0SoIrO2r6B8',
manuallySubmitted: true
}The key properties we’ll be paying more attention to are:
- The tags array, which will contain all tags set during that particular conversation. You’ll have to take into account that if the operator sets the same tag several times, that’ll be shown here. Also, their case is not normalized before sending, so we’ll have to do that ourselves.
- The Items array, which is where the conversation takes place. And now you can see why we may want to clear it up a bit. If we just copy & paste that into the transcript field of our pages, our CMS operators will hate us. No worries though, making this human-readable is quite easy.
- The ID field. This is a unique conversation ID we’ll use to create our page, that way we can be sure we don’t accidentally create two pages with the same transcript.
The rest of the fields can be ignored for now, since they are either irrelevant or optionally filled (like the full name of your user).
Here is the actual code
The code that takes care of the POST request is the following:
app.post('/olark-hook', async function (req, res) {
/**
* 1. get olark payload
* 2. check for tags containing "faq" tag
* 3. create a new FAQ item in butter
* - Title: Olark Question: ID
* - Question: empty
* - Answer: empty
* - Transcript: full convo transcript
*/
let body = JSON.parse(req.body.data)
if(body.tags.map( t => t.toLowerCase() ).indexOf("faq") != -1) {
let pTitle = "Olark question: " + body.id
let newPage = {
title: pTitle,
slug: slug(pTitle),
"page-type": "faq",
fields: {
"en": {
question: "",
answer: "",
transcript: getConvoTranscript(body.items)
}
}
}
let postURL = 'https://api.buttercms.com/v2/pages/'
request.post({ //perform the page creation
url: postURL,
headers: {
"Authorization": "Token " + WRITE_TOKEN
},
json: true,
body: newPage,
}, (err, resp) => { //We're done!
if(err) {
console.log("There was an error: ", err)
return res.json(err)
}
console.log("Done, new page successfully created in ButterCMS!")
res.json({ integrationUrl: "http://your-host.com/olark-hook"})
})
}
})A few things to make a note of:
- The first thing we do is check for the tags. We do that at the same time as we normalize their case. It’s a silly one-liner with a map and an indexOf at the end of that array.
- I’m using the slug npm module, just to cover the basics of generating a slug. Nothing fancy but useful.
- You’ll need to have a valid Write API Token from ButterCMS in order to perform the request, as shown here.
- And finally, the response format is what Olark expects, an integration URL to double-check that the request was valid. If you don’t reply with a valid HTTP response, it’ll assume things aren’t integrated and it will disable your hook, so make sure you do it right.
The function to clean up the transcript is the following:
function getConvoTranscript(items) {
return items.map( i => {
let transcript = []
if(i.kind == 'MessageToVisitor') transcript.push("> ")
else transcript.push("< ")
transcript.push("(" + i.timestamp + "): ")
transcript.push(i.body)
return transcript.join(" ")
}).join("\n")
}It basically takes the array of JSON objects and turns it into a string of lines starting with “<” when the message is from the user and with “>” when from the operator.
Finally, there is one minor detail I overlooked and has cost me a few less hair in my head. Even though Olark’s documentation says they POST the payload in JSON format, they’re actually sending a form-data request. So if you were thinking of using Express BodyParser like this:
app.use(bodyParser.json())you’ll find that your request.body object is actually empty. Instead you’ll have to setup your middleware like this:
app.use(bodyParser.urlencoded({extended: true}))With that, you should be able to make your own web server and hook-compatible script. If you want to, you can check out the working code over here.
Adding the FAQ Page Type in Butter
Notice from the code above how we’re creating a new page of type “FAQ.” This is a Page Type you need to create in ButterCMS.
Here is how to quickly add the Page Type to your Butter account.
First, you need to create a page that will act as a template for your type.
To create a page, simply click on the Pages menu on your left, and then click the New Page button, which you can find in the far right corner.
Once on your page creation screen, simply add the required fields. In our case, we’ll add the question, the answer, and the transcript fields. The first one being a short text field, and the last two being long text fields because we might want to have a long paragraph or two where we can properly answer the question.
Notice how there are a lot of different field types to choose from. We’re keeping the page simple here, but you can go as detailed as you need with different field types (For example, you could add a date field for recording the creation date of the page if that's important for you).
After clicking the Save button, you’ll be redirected onto filling in the actual content of the page, including the page’s title.
Once you click “Save Page Metadata” your page with required fields has been created! You don’t need to fill in the fields, since these are dynamic, changing for each chat that is tagged. The final step to make this single page into a Page Type template to be reused repeatedly for saving all tagged chats is to simply click on the ellipsis button on the far right and then select the “Edit schema” option.
This will take you to edit the page’s fields but with a new blue button, which you need to click: “Create Page Type”. Doing so will ask for you to name your new Page. Simply enter “faq” and click “Save as Page Type”
Once you do that, you’ll be able to create new pages by following the same steps, but you’ll be shown a new type: “FAQ”.
That’s it, now you have the FAQ Page Type we need for our code to work!
Possible enhancements
That is it for this code. With the above notes and following the process I’ve outlined, you should be able to automate the communication between Olark’s chat operator and your ButterCMS users.
It would be amazing if we could pre-populate the “question” and “answer” fields on our pages, since right now, once created, they look like this:
And it is up to the ButterCMS operator on your team to read the transcript and populate the fields above.
I have two possible enhancements that you might want to consider (especially if you have access to dev resources):
- You can use some NLP libraries to analyze the dialog between user and operator and pick up on particular keywords being mentioned. This is interesting if you want to start automatically tagging your questions, which can be helpful if you start having a lot of them, then you can implement a search or categorization feature based on that. NLP.js would be a good match for that.
- You could potentially use a third party chatbot service to pick up and retrieve the actual question being asked and the answer, in order to pre-populate the fields as I mentioned above. This would require quite an extra effort and it would not yield perfect results, so you would still have to manually validate the results, but it could be a fun project to test!
- Finally, one problem with the above flow, is that users can re-open conversations if they want to keep talking before the timeout happens. If they do that, the conversation ID will be maintained, and the script above will not be able to create a new FAQ page with the same ID. So a possible solution for this, would be to keep track of conversation Ids received on your script’s side (maybe a simple database, or a Redis instance. Or even an in-memory cache would suffice) and if the ID is repeated, instead of creating a new page, patch it with the new transcript.
With these three improvements you can have powerful FAQ automation between the two teams and help everyone get back to more important things!
Final words
Olark provides a great deal of help when it comes to contacting your customers one on one, even if it has to be through a chat box. They already have an amazing array of out-of-the-box integrations ready for you to test, but with the help of Webhooks, we can expand that even further. Just like we saw here, helping you integrate with an amazing headless CMS.
such as ButterCMS taking the concept of Content as a Service to the next level.
Let me know in the comments if there is anything I missed or if you have any other ideas to further improve this integration!
Some other use cases for Butter's WRITE API: