19 Best UX Practices for Building Chatbots
How do we make chatbots usable? What we learned from testing various chatbots.
Part 1: 19 Best UX Practices for Building Chatbots [You are here.]
Part 2: 11 More Best UX Practices for Building Chatbot
At Sabre Labs, we explore how exciting, up-and-coming technologies impact travel. Lately, one of these technologies has been conversational interfaces. In 2016, we jumped on the chatbot bandwagon with enthusiasm.
Through our research and prototyping, it quickly became clear: Best practices for good UX in conversational interfaces are still yet to established.
Usability practices for graphical interfaces have been long standardized; anyone looking to improve their websites are pointed to trusted guides like Steve Krug’s Don’t Make Me Think containing well-established usability practices. Unfortunately, for chatbots, these guidelines are only sometimes applicable, and often irrelevant.
In hopes of establishing some usability guidelines for chatbots, I conducted guerilla user tests at a local university in Dallas, asking a total of 15 users to interact with one of three travel-related Facebook Messenger chatbots: Skyscanner, Hipmunk, and Kayak.
For each test, the users were asked to to 1) book a flight, 2) change the destination, and 3) change the departure and/or return dates.
Below are the key findings, observations, and themes that came out of the guerilla testing.
1. Strive for consistency: don’t create expectations that the chatbot can’t meet.
The number one rule of Schneiderman’s “Eight Golden Rules of Interface Design” still applies to chatbots: Strive for consistency.
Seeing that the Skyscanner chatbot accepted weekdays as inputs, a user typed in “Wednesday” as his departure date, which was successfully parsed to the correct date.
Emboldened by his previous success, he then typed in “Monday” when prompted for the return date, only to be met with an error response. He was bewildered by this inconsistency — why was his return date invalid, when it followed the same format as his departure date?
He would’ve had a better experience if he had never been given this expectation. Rather than attempting to give an illusion of high-level NLP (natural language processing) and then failing, you should set correct expectations that match your chatbot’s abilities.
Functionality trumps all. If the chatbot can only partially parse certain formats, forgo the format altogether, and set expectations and provide examples of formats that your chatbot can process.
2. Strive for consistency: provide a way back to the “main menu” somehow.
Many users would complete a task effortlessly, and then ask, “So… where do I go from here?”
In traditional GUIs, it has become second nature for us to look for the home button whenever we want a fresh start. For conversational interfaces, the solution is still forming.
Is it simple as just training the user to say “help” or “start over?” Will this prompt be part of the onboarding process, or at the end of an interaction? Or will the solution not be conversational at all, but simply another button on the interface? Kia Niro took an interesting approach to the onboarding process by using the carousel to explain how to use a Facebook Messenger chatbot.
Regardless of the approach, the user should know how to get to the main menu and how to reset the conversation at any time during the interaction.
3. (Outdated) Until it becomes an established practice, don’t rely on Facebook’s persistent menu.*
Facebook preemptively formulated a solution to the above problem: the persistent menu.
The persistent menu, in theory, is everything a lost user needs: it’s always there, it communicates the basic capabilities of your bot, but it’s also behind a “3-caret icon on the left of the composer,” which is just a fancy way to describe a hamburger menu.
While following platform guidelines is critical to create consistent experiences, I can’t advocate for the use of Facebook’s persistent menus… because no one clicks on them.
Out of 20 users we tested, only one even thought to click on this hamburger menu. The information icon on the upper-right corner got more attention than the hamburger menu ever did. Three people hovered over the helpful looking “i” on the upper right-hand corner when they got lost, but the hamburger menu never even registered to these observant users.
Anti-hamburger menu-ers, here’s example #1001 of hamburger menus failing.
*Facebook has since updated this functionality, but this is still a good example of why you shouldn’t hide critical functionalities.
4. Don’t make the users go tl;dr.
In its chatty excitement, the Hipmunk chatbot hits the users with four opening messages, totaling nearly 350 characters, which most users scrolled right past.
In general, anything more than three lines of text seemed to activate the tl;dr (too long; didn’t read) response in users.
How long your message will seem will depend on the screen size of the device, but a good rule of thumb is to have no more than 90 characters per message (around three lines on mobile).
Having to scroll up to see content is an instant trigger for tl;dr (too long; didn’t read). Try to keep your message to less than 63 characters (around two lines). Multiple messages shouldn’t add up to more than ~140 characters at a time.
5. Test your chatbot on different devices to make sure users are not scrolling to read.
Hipmunk sent an image in addition to its lengthy intro, making users scroll up to read its messages and all the buttons it offered.
Test your chatbots on different devices. One of the widely-touted “pros” of chatbots is that the frontend does not need to be designed — but they are still being displayed on a screen. Testing the functionalities across various screen sizes is still very necessary, and a quick way to gain usability points.
NOTE: Keep in mind as well that users will have their keyboards pulled up on mobile. So realistically, you only have half the phone screen as your playground.
Don’t send multiple messages that will not fit in the display of a mobile phone. This makes the users have to scroll up to see content and is a surefire way to make them glaze over.
6. Use UI components as “guard rails” in a conversation.
The beauty of conversational interfaces is its flexibility. The paths that the user could theoretically take are virtually infinite. Unfortunately, natural language processing is currently nowhere near being able to handle this.
Buttons and quick replies are almost integral for keeping users on the happy path and away from too many apologetic error messages.
They will also help to prevent users from encountering the dreaded “blinking cursor” scenario, where users are faced with an empty text field and are unsure of how to proceed.
Watch out though, these guard rails can limit our users if we are not careful…
7. WARNING: Buttons and quick replies give an illusion of limited choices.
For better or worse, when users are presented with UI controls, they will not think to do anything other than to interact with the given controls.
This was a big failure point for Skyscanner. When users were asked to change their departure date to one that fell outside the two button options, they were simply… stuck.
When asked why they had not typed in the date instead, most users replied that they had not thought to do so. Others replied that they weren’t sure if the chatbot would be able to process their text input.
Most users will no longer perceive sending a message as a viable action when a chatbot offers buttons or quick replies as a way to respond.
This is a natural response to being given a set of choices that is probably exacerbated by the fact that chatbots are a new technology. Users are mistrustful of new technologies, and this mistrust translates into safe choices.
Users are mistrustful of new technologies, and this mistrust translates into safe choices.
Thus, UI components in chat is a double-edged sword. They are great for keeping users on a happy path, easing them into a new technology, and easing the mental and physical load during interactions. However, they can also easily trap your user by creating an illusion of limited choices.
If your chatbot utilizes buttons or quick replies, include an option to nudge users into typing a response if needed, as Expedia did.
8. Make interactions predictable.
Payments in Facebook Messenger is still in beta, so most chatbots involving shopping eventually redirect users to their respective websites.
One of the biggest problems with these redirects was that it was impossible to distinguish which buttons would result in a redirect and which buttons would continue the conversation in Messenger.
One could argue that the blame is on Facebook for using the same button UI for two distinct kinds of interactions. However, it’s our job as designers to work with the platform and make sure that we give users the best experience possible. Unpredictable interactions will result in distrust and users will be discouraged from exploring and discovering your chatbot.
Make sure all interactions are predictable. If clicking a button opens a new page, the users should be made aware, most likely through the button text.
9. If you must redirect users out of Messenger, use webviews.
Facebook offers webviews and extensions, allowing “custom experiences that still appear to be part of the Messenger thread.”
Webviews pull up over your conversations, mitigating the disrupted flow by giving the impression that the conversation is being continued inside the Messenger app.
If you must redirect, use webviews to make for a smoother conversational experience.
10. Make your redirects relevant.
It was astonishing how many Facebook chatbots redirected to a booking page without maintaining any of the user entries from the chatbot.
If the user just spent five minutes filling out information on your chatbot, that information needs to be absolutely be persisted in the redirect. Do not inconvenience the users without being able to offer something in return.
The number of chatbots that did this was so high that it makes me think that it may have been a technological limitation at the time, which makes me wonder, should the option have been presented at all in the first place?
Any information the user provided the chatbot with should be persisted in the redirect. If this is not possible — reconsider offering the option at all.
11. Know when to keep context, and when to reset context.
Context-keeping is great — it’s touted as one of the winning features of conversational interfaces. However, inappropriate context-keeping made changing dates completely unintuitive in Skyscanner.
Skyscanner had a bug/feature that made changing dates virtually impossible: it retained the previously inputted dates when users tried to modify their search.
In the below example, the user had first searched for a flight departing on December 16 and returning on December 18.
I asked him to change the departure date to December 20 — a date that was later than the return date of December 18. Skyscanner would throw an error, as the “return date needs to be after the departure date.”
Everyone was bewildered by this error message. In the context of the conversation, users thought that Skyscanner had become confused and was trying to change the return dates rather than the outbound dates.
What was really happening was that Skyscanner had not reset the previously inputted dates, but this oversight resulted in much confusion for the users.
If you’re saving previously inputted information, makes sure that information is being retained and reset at the right moments.
12. Wording matters: Clarity over elegance.
As you may have noticed, instead of using the more common terms, “departure” and “return,” Skyscanner opted to use the terms “outbound” and “inbound” to describe dates.
The words “outbound” and “inbound” may be more aesthetically pleasing than “departure” and “return.” Unfortunately, the similarity between words being so similar led to a lot of confusion for the users. It seemed to lead to Semantic Satiation, where words lose meaning with increasing repetition.
This became especially problematic when the users attempted to change flight dates. Users would also mistake the meaning of one word for the other.
Chatbots that used common, distinctive phrases like “departure date” and “return date” did not face this issue, confirming my hypothesis that the issue was due to the the diction.
Use distinct words that you would use in conversations. Clarity wins over elegance, every time.
13. Wording matters: Confirm by asking, not stating.
Say that a user is trying to book a flight next Monday, but the chatbot interprets it as this Monday, and states, “You’re booking a flight this Monday.”
This statement is less than ideal for a couple of reasons:
- Users may think that there is no option to correct the mistake
- Users may feel that they made a mistake by saying the wrong thing
Instead, a better way to confirm would be to ask, “Is it correct that you want to book a flight this Monday?”
Formulate confirmations as questions, not statements. Questions are a natural segue into error corrections and removes the blame from user.
14. Wording matters: The question will form the user response.
To determine the departing airport, Hipmunk will ask, “What airport do you usually fly from?” Most users did not have trouble with this step, but one user replied with “american airlines.”
This prompted Hipmunk to ask again, “Where are you flying from? Say something like “from SFO.”
This simple change in wording resulted in the correct response, “DFW.”
How you formulate your questions will affect the user’s responses. Although both questions had the same meaning, the use of “what” versus “where” resulted in two completely different answers.
If Hipmunk had used “Where are you flying from as its initial question, I believe the reprompt would have never been activated in the first place. This goes back to the point of users not reading — most are constantly scanning for words that pop out at them.
Unsurprisingly, words matter a lot for chatbots. The importance of thoughtfully selected diction cannot be overemphasized, especially keywords and phrases that can greatly influence the user’s understanding of the conversation.
Carefully consider the words you choose, especially for points of interactions. Particularly, be cognizant of the leading words such as “when,” “where,” and “what” for question.
15. Don’t leave the user hanging.
In traditional apps and websites, users have patience for only a few milliseconds before they think something is wrong. Designers have dealt with this challenge using loading icons and messages for applications that take more time.
We currently have a lot more leeway with chatbots since users currently do not have expectations for proper chatbot response times. Instead, user experience in messaging interfaces appears to be subconsciously informed by texting with other humans, which always has some amount of lag time.
In my user studies, I found that 8 seconds is about how long a user will wait until he or she starts trying to reboot the “unresponsive” chatbot.
Regardless of this generous leeway, users should always get hard feedback that 1) the chatbot is processing things and 2) a response is on the way.
Facebook has typing indicators, which are useful for smoothing out experiences for short wait times, but they are automatically turned off after 20 seconds.
If you anticipate that it will take longer than a few seconds for a chatbot to reply to something, let the user know upfront, and monitor the process to let the user know that things are taking longer than normal.
We already do this with loading messages. For instance, a lot of apps say something like, “Things are taking longer than usual... Try refreshing?” Refreshing isn’t an option in chatbots, but you should still keep the user informed of the process going on in other ways.
The user only knows of what you tell them. Be proactive about loading messages. Don’t make users wonder if your chatbot is broken.
16. Give opportunities to correct errors throughout the conversation.
Neither chatbots nor humans are perfect. Undo and cancel are essential functionalities for a smooth error-correcting experience.
Until undo and cancel as a functionalities are implicitly understood across multiple conversational platforms, users will not know to call upon functionalities exist unless they are introduced to them. However, once they know this option is available to them, the users will feel that much more secure exploring the chatbot.
Allow users to go back, cancel, and undo. Be smart about informing users about these options at critical points of the conversations, so that the users know how to access those options.
17. Don’t display important texts as captions.
Facebook offers the ability to place captions on images. Images are eye-catching; captions are not.
We’ve been trained through years of reading to regard captions as unimportant. The likelihood of captions getting read is even lower, and as we’ve discussed, people already don’t read.
Don’t do what Expedia did — display critical onboarding information as a caption, which also happens to be set don’t-read-me-gray.
Important details should never be displayed as captions.
18. Keep the humor safe and professional.
There is a fine line between cute and annoying, or even dangerous… like this message Hipmunk sent.
Please don’t joke about charging my credit card. It’s one thing to be funny, but find ways to do it without risking user trust.
This article is part of Sabre Labs team’s analysis on emerging technology trends impacting travel. For more insights, download the Emerging Tech in Travel 2017 Report.
Please reach out to Sabre Labs at sabrelabs@sabre.com for further inquiries or discussions on the future of technology and travel.
