Helping Your Baby Bot Learn To Chat Like A Grown Up Bot

In a conversation the user is always right – what are you doing to keep improving your bot’s understanding? Baby bots need help to grow up.

When things go wrong in your chatbot it will feel to the user not that the application just isn’t working properly, but that they are not being listened to. Getting it right when things go wrong is critical for good user experience.

In this article we’re going to focus on chatbots using Natural Language Understanding (NLU) since they have some interesting challenges around ‘error handling’, and as technology advances more users will expect NLU to be the norm.

Below are some guidelines and tips for guiding users when things go wrong, and some advice on how to take what you’re learning from your users and make your chatbot even better.

Data Input Validation Errors

The patterns of how we handle for unexpected or incorrect data in a chatbot is different to other interfaces, so if your chatbot collects data this is a key part of your design.

We’re all used to the red errors on websites when we ‘get something wrong’. Validation errors on website are usually quite aggressive — stern direct commands, red text, highlighted fields. This may be useful on a full screen form, but not appropriate when you are filling in information through a conversation.

When dealing with validation errors in a Conversational Interface you should assume the user is never wrong, and the failure has been in your understanding of what they said. This is a conversation — if you were talking to someone who kept abruptly correcting you, you wouldn’t want to keep talking to them for long!

You also need to be very aware that the conversational tone and interface may lead your users to type things they never would have considered typing into a box on a website. Below is an example of what a conversation would look like if the same error messages that were used on this particular website (not one of mine!) were put straight into the chatbot:

On a website you would never expect someone to type a sentence in when asked for an email address, so you don’t need to spend a lot of time considering how you would deal with that. Validation errors on a website can be very specific to the formatting or missing information. In a chatbot you need to consider that at any point the user could actually be asking for assistance instead of giving you the information you asked for

Bot: Hi there! To start the process I’ll need your account number.
User: Oh no, I’ve forgotten it!

If the next thing you say is “Account numbers should be between 8–15 characters and contain only numbers” that is really unhelpful. Recognising this can be difficult. However, if you use the rule that the user is always right this will guide you in appropriate responses.

Bot: Hi there! To start the process I’ll need your account number.
User: Oh no, I’ve forgotten it!
Bot: I’m sorry, I didn’t seem to get your account number there — do you need help finding it?

That response isn’t perfect, but it could be used in a variety of scenarios where you didn’t get something that matches the format of an account number for a user and aren’t able to offer more specific guidance.

So what can we do?

  1. Make sure to build in really good help messages. The user should never feel like they are helpless or being ‘told off’. You don’t need to understand and respond specifically to everything a user could say (although if you can that’s awesome) as long as your general error message is broad enough to assist the user in a variety of situations.
  2. Try to identify things users regularly have trouble answering and proactively offer assistance.

Bot: Hi there! To start the process I’ll need your account number. It’s found on the top of your bills. Say ‘help’ at any time if you need more assistance.

Note that to do this you need to be able to record and review errors within your system. This gets even more important as we move to the next issues.

General Natural Language Understanding Failures

It may seem obvious, but with Natural Language Understanding comes the challenge of understanding natural language. Humans can often not be great at this. Computers are definitely worse!

For today at least, it’s not possible to create a chatbot that can understanding anything anyone could say — there are so many different ways to express the same thing, and an almost infinite number of things to say. This makes it really important that we have good error handling paths in our conversation flows, as this should be considered a normal part of the flow rather than the ‘exception’.

What can we do?

As above we need to make sure good set of generalised error messages that can be used across the chatbot when we aren’t able to match with the customer said with anything. Again, remember the user is always right — they haven’t said the wrong thing, you just haven’t understood.

For generalised error messages that aren’t data input:

  • The message should be clear that you didn’t understand and will need the user to re-ask in the different way.
  • It’s fine to be a little bit interesting or amusing (“I’m just a baby bot — I don’t know how to do everything yet — I’m working on getting better”), but be sensitive as well, as the user may not be messing around (as I often do to test bots out), but genuinely trying to be understood.
  • Have a selection of responses that can be returned so that the user doesn’t just see the same one over and over. Considering refreshing these regularly.
  • Try to guide the user back to the expected conversation – offer them an alternative thing to say to get back on track, ideally something relevant to where in the conversation they are, but at least offering a phrase that can be used to get help.
  • Once you have failed to understand a number of times in a row, give the user an action that they can take so they aren’t left feeling helpless (“send feedback”, “report a problem”). Or if you can, hand over to a human, and bring whatever context you have so that the conversation can just keep going, not fully restart.

Where not understanding gets worse…

Much worse than not understanding at all is your bot thinking that it has understood when it didn’t. This is very hard to automatically detect, and, as I have found from our own bot, massively more frustrating to users than not understanding what they said at all. Users complain to us far more frequently about misunderstanding than not understanding.

User: Can I order a lemonade?
Bot: What size pizza did you want?
User: No, a lemonade
Bot: I’m sorry — I don’t understand. Can you try asking in another way?

So what can we do?

There is no magic here — this is going to take some good old human intelligence to improve. To be a good support system for your bot and help it get better you will have to put some effort in, and ensure you have logged conversations somewhere (ideally anonymised) that you can review.

  1. It should be easy to automatically get a list of the things the bot didn’t understand at all — get it, and look at it.
  2. Look at feedback from users for complaints about mismatches.
  3. Review at least a subset of the logs of things the user said that the bot believed it matched — check for things that were matched incorrectly and make a note of these. You can also use sentiment analysis to find instances you should review, but note this won’t be perfect either.
  4. Take the missed or mismatched conversations and use them to adjust and retrain your NLU model to improve your matches. Easy to say, but difficult to do — this can sometimes feel like a dark art, but it’s worth it to make sure your bot is improving.
  5. Think about improving user experience even for users deliberately going off topic (trying to test the limits or be funny). If you regularly see the same off-topic conversations you can build specific conversations in for those scenarios. In our internal Employee Digital Assistant bot users kept asking it to tell them a joke. It was a quick change to add in recognition for that and respond “I would love to — but shouldn’t you be working?”. It was well outside the intended function, but adding that recognition made it a little more ‘fun’ (one user commented “well played chatbot”). It is important that you never compromise core functionality to do this, so choose this path carefully.
  6. Think about the wording of the ‘successful’ responses you give. Could it seem like the right answer, but actually mislead the user when a mis-match has happened? In the absence of a graphical UI it’s good to be explicit and repeat back information. For example instead of:

User: I’d like to order one lemonade please
Bot: Ok, that’s ordered for you!

It would be better to use something that explicitly calls out what has been ordered for the user:

User: I’d like to order one lemonade please
Bot: Ok, I’ve ordered one lemon cheesecake for you!

The user may still be frustrated, but much less than they will be when the lemonade they believe is ordered doesn’t show up. This makes it clear immediately that something has gone wrong, and hopefully can be corrected. The less free cheesecakes you have hand out to appease angry customers the better.

Bot: Ok, I’ve ordered one lemon cheesecake for you
User: That’s not right!
Bot: Do you want me to cancel that?
User: Yes
Bot: If you need some help with your order I can hand you over to one of my human helpers. Would you like me to do that now?

Be kind to your users, be on their side, take responsibility for what you didn’t understand and always give them a way out so they aren’t stuck in a conversation that has gone wrong.


Handling conversational errors is hard — you can’t just use the same patterns you had on your website. You need to expect the unexpected and be ready to put some effort into improving your bot over time. Put the effort in however, and you won’t need to hand out free cheesecakes to angry users who won’t be coming back.

This article is the second part of a series —

Clap 👏 below to recommend this to others. You can also follow me here or on twitter @virtualgill.