Creating a Chat client with AppSync (and adding Bots!)
Because sometimes human beings also want to chat.
How to build a chat where humans talk to each other along with bots where frequent disconnections occur using Amazon AppSync and GraphQL.
At Conversalab we have a lot of experience in the market developing Bots for Telegram, Messenger, Bot Framework and also virtual assistants on the web.
This time the challenge was to include us (human beings) with our dialogues on the same platform as the bots. With some unique restrictions.
The problem
Due to the nature of the business linked to logistics, some particular considerations and conditions arise.
The messaging service to be used should be distributed, fault tolerant, usable even in the event of disconnections (very common in rural environments), used in a mobile application and that does not require a social login (this was not negotiable).
By business rules we could not force our users to use personal social networks (Telegram, Whatsapp, Messenger, Skype, Skype for Business, Microsoft Teams).
Additional restrictions included the need to be able to handle huge volumes of messages. The interaction should be heterogeneous for person-to-person dialogues, as well as person-to-bot.
We also needed a web client, a native Android client embedded in a client’s own application and, of course, our bots answering from their own server.
Possible solutions
We evaluate alternatives present in the market and find candidates in two groups:
The chats designed to work person to person.
The solutions between bots that we had already used in the past.
However, the state of the art did not allow us to find satisfactory hybrid solutions.
A possible solution
After asking in forums and specialized sites and doing research on possible architectures we decided to use AppSync technology.
This solution allows keeping certain information synchronized in different environments and is specially designed to handle small transactional data, resolve synchronization conflicts, and be agnostic with respect to data traveling between devices using a declarative language called GraphQL to predicate about objects and operations.
Amazon Web Services implements AppSync since it was introduced at the beginning of this year (even one of its promoted uses consists of a chat that we use as a base to build our messaging approach).
Amazon AppSync is designed to handle large volumes using a NoSQL database . In all our previous projects we used relational databases processing more than 50,000,000 messages and decided to scale in another way given the experience acquired.
The implementation
To implement the messaging layer, we decided to test the demo that AWS provides which served as an excellent base to evolve our own service.
The domain of the problem required, as stated above, that a web client coexist (with good connectivity) with an Android client (with frequent disconnection periods).
The web client provided in the demo turned out to be very basic software (with very little functionality) and at the same time excessively heavy. The choice of Node.JS with Angular is not the best when it comes to thinking about lightweight pages.
The chat in the web client represents only a small fraction of the functionality requested by the client and as help for the daily operation of business rules so we could not afford to penalize our operators with a heavy customer just in case they needed to dialogue with other people.
The solution we adopted was to port the Microsoft webchat that we had already used in production for several years) to which we added some features requested by the client such as: visualized message thick, autocomplete typeahead and a more modern look and feel.
As for the Android client we noticed some immaturity in the implementation and problems of memory leaks in the subscriptions, so we chose to use only part of the functionality (omitting the subscriptions and changing the push model by one of pull as we will explain later).
On the server side everything runs very smoothly. Messaging is persisted in DynamoDB tables (the AWS NoSQL solution) with provisioned capacity and automatic scaling.
The scheme of tables that we use is very similar to the one that brought the ChatQL demo with some extra business rules.
When a new record arrives in the DynamoDB table of messages, an event is triggered in a Lambda function that determines if the message must be answered by a Bot. In affirmative case the Lambda function invokes our bot server. The response is persisted in the DynamoDB table, so all clients are updated. To avoid a deadlock, very clear rules are used to trigger events and we add a second line of defense to detect possible deadlocks in case these rules fail.
To track and debug messaging, we use Cloudwatch on lambda functions. In addition to the messaging function already described, we use extra functions to register new devices and new conversations.
Evolving the business scheme
The data transported by AppSync are defined in a GraphQL scheme. We found it extremely easy to make a change in such schemes using specific resolvers for each necessary message. The flexibility of this service is key to be able to monitor the evolution of the business.
Overcoming the problems of a new technology
After implementing the complete messaging circuit, we noticed that the libraries provided by AWS for AppSync synchronization were not robust enough for an output to production. We discovered that they consumed too much memory and crashed frequently.
By isolating the problem we determined that the failures occurred when subscribing to changes in the server. As we could not do without such functionality we chose to replace it with a pull solution using polling and piggybacking with the news in the messaging.
Satisfied quality attributes
As a summary of our solution we will indicate how the needs raised at the beginning are met:
Ability to respond Offline : AppSync is responsible in a transparent way to reconcile and synchronize the data (in this case the messaging and conversations).
Real-time Response : The latency of the service turned out to be unbeatable. Using websockets we noticed that the conversations are in a real time manner as long as the connection is good.
High availability and Efficiency in the transmission of contents : The scheme of the NoSQL databases is unstructured. This means that in order to transmit an image or a video within a message, an option is to embed that content directly. If we had used a normalized relational database we should have indicated a link to said BLOB or a link to an external image. This is an excellent solution for our scheme where the device moves between areas of great connectivity (where the complete resources are downloaded) to spend long periods without connection.
Centralization of the business logic : All the logic is solved in the server avoiding having to keep duplicated the business rules in the devices and / or in the web clients.
Modifiability : By defining the messaging in a declarative way (and hiding the implementation in the resolvers) it is very easy to make changes and evolve together with the business.
Conclusions
Building a chat with bots from the beginning seemed a challenge already solved by 2018. However, there is still a lot of space to create robust solutions for the market.
