Better Programming Methods For Chatbots
In this story we will be talking about methods of developing chatbots and deploying them. Bear in mind that most of this article will directly concern NodeJS development, and some other languages may have other methods more adequate.
Disclaimer : These “best” methods are chosen from experimentation and personal experience at Bottomatik. They might not be suitable for your specific needs and this article only aims to give you ideas to better program your bots.
Choose the right API 🏹
Most conversational interfaces offer what we call WebHook APIs. These APIs work by sending a request to your server based on an event on their side. In this case that event can be a message, the click of a button, or an entry point to the chat.
When programming a chatbot be sure to choose the API you are most comfortable with and the one fastest to develop in. For example, Slack offers a real-time API (based on WebSockets) and a WebHook API.
Essentially they are the same, each one sends a request to your server when an event happens. However, in a WebSocket API, you’ll have to “route” everything yourself — checking every message — where as in a WebHook API, you can choose any HTTP Server framework to help you route your server.
Other than that, in a Socket-based API, you have to be aware that the socket might close, and you need to re-open it. If this happens you might lose the “real-time” part of the API, since it will have to store the message to be sent to you once the connection restarts.
Choosing the right one will depend on your needs and what you have to build, real-time can be very benefitial, but a bit more complicated to implement.
📚 Build or pick a framework ?
When developing chatbots, unless you plan on programming just one, you will have to choose whether to make your own framework or to choose one. There are many bot frameworks available there and you can choose the one that best fits your needs.
Building a framework is slow, but the payoff can be huge
When choosing whether to build or pick, you have to weigh the consequences, for example:
- Building a framework takes time
- Building a framework is hard
- Building a framework needs a team (small)
- Building a framework needs planning
- Picking a framework means bending to its rules
- Picking a framework means sacrificing some functionality
- Picking a framework means sacrificing some interfaces
At Bottomatik we chose to build our own framework. Yes it does take time, and we have to constantly update it. But it gives us extreme flexibility, and we can add or remove interfaces and functionalities as we like.
This also means that we always have a developer in hand for fixing/improving the framework, and thus less workforce for the actual products. Also, developing a framework means choosing the right APIs for easier integration.
Use events (or simply EDD) 🎫
Nomatter what language you are developing in, use events. Using events ensures you write code simply and in a structured manner. You can use events like ‘message’, ‘postback’, ‘entry’ or others. Look at this example:
Using events you also encapsulate your functions (with lambda in Python, Runnables in Java and simple functions in JS) and ensure less bugs will come from misinterpreting user ids, user messages or other variables.
This will ensure clean, comprehensible code. However you might find it difficult to build chat flows using events. It will also allow you to write asynchronous code, improving the responses time of your chatbots.
Use a load balancer 🚦
Chabtots are extremely easy to access. You open an app, tap the name and that’s it. That means that you will probably be getting a lot, and by a lot I mean A LOT, of messages.
By experimentation we have been able to get more than 1000 messages/s without a load balancer on NodeJS (using Nginx as the front server). This is ok for our sector of work, indeed we are not expecting more than 1000 m/s, which is a lot.
However, if you are building a public chatbot, specially if it’s Global, you could be getting a lot more than that. Imagine you’ve got a million monthly users, and you release a new functionality. You can expect your bot to get a lot of messages.
Using a load balancer is extremely easy, you setup your front server (Nginx, Amazon, Apache, whatever…) and your machines and see them run one against each other in a race to the fastest response. If you are using NodeJS you can even go further and use Clustering on your app, to build a load balancer on the listening port and for each CPU core.
Here you have an article from Nginx explaining how to make a load balancer for NodeJS.
You’ll need a fast database 💨
Here you have A TON of options and it depends on your preference. Do you want a distributed DB ? Do you need a NoSQL DB ? Are you used to MySQL or Postgres ?
Whatever the case, you’ll need it to be fast. You will get a ton of requests at the same time, and in a chat based application people expect fast responses. One second is just acceptable, if you can do it faster, please do.
If you are using an SQL database, you can use any ORM of your choice. Python’s Django has one, there is Sequelize for Node, and a ton for Java and PHP. When using SQL be sure to use transactions to send more requests at once and ensuring faster response times.
Using a decentralized DB can be a good idea to improve response times. The only caveat is that you may not have up-to-date data all the time. If your bot can survive with it, so let it be.
Here at Bottomatik we’ve chosen a RAM database built by our own hands. By doing this we ensure fast transactions and we are free to change it as we wish. However the maintenance is very hard, as it is a NoSQL database built by our own hands, IPC is complicated and ensuring it’s integrity takes a lot of designing and testing.
🤖 Use caching for AI
Ok, so this is probably the most important point of this article.
When using AI or ML for your chatbot, these algorithms take time and delay your cahtbot’s response. It’s just the way it is, machine learning algorithms are slow, specially if poorly written or if they don’t have the system requirements they need.
To improve this you can use caching. Let’s say you use an AI server like API.AI or WIT.AI or whatever. Every time you get a message, you perform a request to that server and wait for the response (probably an intent). Depending on the intent, you will query your DB, maybe more APIs, wait for the responses and build the message. OMG that was long.
However, you can set up a simple word comparator to cache the AI response. It will not be 100% accurate, but it can get the job done. You only need to store the words associated with every intent and compare it on the cache.
This can also be approached by writing a ML algorithm based on the kNN method.
In any case, you must not make it slower by adding every word or every message to your cache. You can setup a “closeness” function that tells you how close to the intent you are, and then decide to query the AI server or not.
Conclusion
Although this article can seem a little bit boring, since we explicit all the same needs than any other web app, you have to keep in mind that a chatbot can be getting a LOT MORE requests than a web app.
A wep app will get tons of different requests, for files and API. But your bot will be getting the same requests over an over, and has to compare every text it gets with the AI to get the response, which is extremely slow and resource consuming.
Bottomatik is a chatbot agency based in France. We build chatbots for all kind of events, specializing in Museums, Festivals, Theme Parks and more. Bottomatik also develops solutions for restaurants & bars. If you want to know more please contact us at hello@bottomatik.com .
