Troubleshooting Facebook Messenger Bots
Best practice for releasing your bot.
We are a small startup building bots. We have several commercial bots running on Telegram, Skype, Slack and now Facebook Messenger.
Mastering a platform is cumbersome. Working with many of them is NP Complete.
We are very excited with the brand new Messenger Platform. Nevertheless, we have faced several issues with the platform itself and with our overall user experience.
We decided to share the lessons learnt with the growing bot developer family.
As a word of caution: Facebook Messenger Platform is currently in beta so the issues described in this article are part of the maturing process.
Here are some of the issues we experienced:
Is Messaging a Super-set of Inbox ?
We’ve been using Facebook inbox API for years. We have been managing several communities using automatic processes that are triggered and respond to messages to the Facebook Inbox. “La gente anda diciendo” (what people are saying) is a collaborative project with over 7 million Facebook fans. Last year we designed a tool for them with which they filtered and managed over 10.000 collaborations sent by Facebook and Twitter.
We read their page messages on demand and also had set webhooks on inbox endpoints in order to be quickly notified on important messages.
On April we attended the Facebook F8 event. Facebook Messenger implementation using a new api and webhooks seemed very straightforward to us. So we began testing and having fun with it.
To recap: The Most Complete Report on What Facebook Messenger has Launched on F8
To our surprise Inbox Messages and Messenger Messages share lots of things but are different in some important ways as we will describe below.
(BTW You can talk to their Bot here)
Are My Webhooks Connected and Subscribed?
Implementing Facebook Messenger was very hard at the beginning. Lots of error prone steps and very few tools to check for mistakes.
Update: Now they made a very useful guide
You must connect AND subscribe your webhooks to your applications. If you make any mistake you get no errors. The bot does not answer (1).
On the early days webhook and page subscriptions could only be managed via Graph.api now they have a graphical interface for that.
Have You Double Checked It?
You must go to your application settings, connect the webhook, answer to its ping challenge in a good way and subscribe your page. If you make any mistake you get no errors. The bot does not answer (2).
Do Not Trust the Graphical Tool
If you subscribe your page to the webhooks using the graphical tool in Developer Application sometimes it just doesn’t work.
Do a manual post to the edge me/subscribed_apps using your page token in the Graph Api. Remember: product is in beta and UIs are very nice but command line is where the magic happens.
Update 2017: Product is no longer in Beta.
Always Be Secure
Your webhook must be set up on a secure server with a valid SSL certificate. If you make any mistake you get no errors. The bot does not answer (3).
How to Kill a Bot with 10 http Requests
Who Speaks For You ?
The user initiating the conversation with the testing bot must have a verified account and a role in the testing application. If your testing team make any mistake you get no errors. The bot does not answer (4).
Is the Start Message the Welcome Message?
Facebook recently added a feature so you can welcome new users and give them context once they start to chat in the same way that Telegram does with its /start message.
Nevertheless users seldom read the initial message. What is more the message stands on the page whether it has a bot or not so is pretty confusing to have a message talking about a bot when the bot does not answer (i.e. is in review process and not available to everyone). As a word of caution set up this message as the last step in you process and also consider Page Messenger chat Message.
Please Test Me Saying ‘Hello World’
So you want to test if the bot answers? Treat him nicely. If you are a programmer your first testing message might be ‘Hello World’. Don’t lose your nerve. Keep treating him nicely even if he doesn’t answer. If you fix all the mistakes you made in the previous mistakes but the bot does not answer you might probably get anxious and start typing shorter messages like ‘a’ , ‘x’ etc. (as lazy programmers like us usually do)
Featured CBM: Testing Chatbots: How to Ensure a Bot Says the Right Thing at the Right Time
Bad mistake! You will be confined to spam inbox (which the bot does not read by the way). Check it out. If you make that mistake you get no errors. The bot does not answer (5).
No Buttons on Facebook Lite or Mobile Site
If you have an older Facebook Mobile version or Facebook Lite (which is very popular on developing countries like ours) or use the mobile site you might notice buttons and structured messages are not shown. Just texts. So how will the interaction be like? How do we test bots on those platforms ?
Facebook does not provide information about the channel where the message was originated so we cannot take even a bad decision with a stinky IF.
Take Poncho as an example. Poncho is Facebook’s bot platform flagship
Facebook encourages us to use structured messages like its fancy carousel. But it well not be displayed to most of my country’s users.
If your client makes the mistake of having a light version. The bot just does not display all the content (6).
Quick Replies (Buttons Reloaded)
On July Facebook released quick reply buttons. A great way to direct a conversation. Once the button is pressed it turns into text (In the same way Telegram does). If you read the conversation it seems as it the user typed the text. This works nice on Messenger app and on messenger.com site but it fails to show on certain Chrome Desktop browsers (if you check your inbox). This might be just an error but it is disappointing to read user transcripts and realize they don’t get the options.
Test Apps Are Great. Can They Talk To Pages?
Tests apps are very useful. We built them on our Android project because the set up is very easy. Our advice is ‘don’t use it now with your messenger bots’.
Test apps cannot talk to real pages. This is something Facebook will fix very soon. In the meantime you should use a mock real application for testing purposes. If you make that mistake you get no errors. The bot does not answer (7).
Update 2017. This issue seems to be fixed
Facebook Inbox or Facebook Messages (Revisited)
Sometimes bots are not very smart. We discover our virtual assistants can solve 97% of user requests but sometimes you need to silence them in order to human assist the user. Therefore: how do we make such? We can build a nice web monitor and give it to our operators so they can stop our bots or we could teach the chatbots to shut up once a human gains control. We decided the later. The rule was simple “If a human speaks as a Page you shut up”. The implementation was not that easy. How does the bot acknowledge the human discourse? Bots don’t get webhook pings whenever a page talks. Let’s make a hack! Let us subscribe to the delivery event and read whenever a user reads a message that is not sent by the bot! So we did it. We got the delivery ping noticing the user has read a message ‘mid.14638151552330:4665f9e66d7ai142a74' That’s all, Folks !! Let’s read the message! Not so fast ! We got Requires read_page_mailboxes permission to manage the object. Which translates to “you are trying to read a mailbox message but you can’t”.
Of course we can’t ! We don’t want to read mailboxes we just want to chat!
Mixed Channels Part Three
We have talked to many of our clients that are retail stores with well established pages (More them 500K users each). Many of them already use the page channel to manage questions and complains as Facebook suggests. We encourage them to build bots. But they are already using the channel so they don’t want to mix complains with new sales. If only they were different channels.
One Shot Greetings and Get Started
Greetings were released in June as a great way of introducing the bot following good etiquette rules. Nevertheless they show up just one (after you talk to the bot they vanish). So they are actually very difficult to test. In fact you have one shot per Facebook Account. So we have to beg our fellow workers to borrow their mobile phones in order to make some tests.
Once They Work They Are Very Fun
In the end if you make no mistakes you get no errors. The bot answers and help your business grow and connects lots of people !
Once its done you have a brand new shining bot.
You can see some examples on the wild here.
Final Note. This Article is Inaccurate and Will Soon Be Deprecated
This is intended to be an ongoing project. Facebook Platform is evolving every day so we hope most of its content will be useless very soon.
This article is meant to help other developers but is based on the experience of just a bunch of people. It certainly has lots of inaccuracies. We encourage to comment on it so we can fix them.
Please don’t hesitate to send us your suggestions, comments etc. in order to make it a good check list. We intend to update this article with new information in order to keep it alive.
For more on UX: 19 Best UX Practices for Building Chatbots
