5 Steps: Automated Testing of Chatbots in Eclipse
So you are a test automation engineer and your test object is a chatbot ? Great. Let’s do it.
UPDATE 2020/11/01: Botium’s free plan is live! With Botium Box Mini you will be able to:
- use multiple chatbot technologies
- set up test automation in a few minutes
- enjoy a new improved user interface
- get the benefits of a hosted, free service
Botium, the Selenium for Chatbots
This article shows how to install, configure and run Botium, the Selenium for chatbots, within Eclipse. As a test automation engineer, you are most likely familiar with Selenium or Appium: Selenium for testing web applications, Appium for testing smartphone apps, Botium for testing chatbots.
Botium-Bindings is the companion library for Botium to provide the link to existing test runners like Mocha and Jasmine. Read this as a starter:
The following steps show how to initialize your Eclipse project. At the bottom you can find link for downloading the Eclipse workspace.
Step 1: Get Eclipse Neon
Download and install Eclipse Neon. It contains support for Node.js.
Step 2: Create and initialize a Node.js project
Create a new generic project in Eclipse. Now use the “npm init” wizard in Eclipse to initialize the package for usage with Node.js. You will have to answer some questions.
Open the resulting package.json file and add scripts and devDependencies as in this gist.
Run the “npm install” configuration to install all the dependencies. Just right click on “package.json”, open the “Run As” menu and select “npm Install”.
Step 3: Add Botium configuration files
Add a file botium.json to the project directory (see gist). It contains the configuration for Botium to connect to a chatbot running on IBM Watson.
See the Botium Wiki for additional information. Botium supports a broad range of chatbot technologies.
Create a directory named “spec” and place a file called “botium.spec.js” in there (see gist). It’s just a small piece of Botium code linking the test cases to the test runner.
Step 4: Define the test cases
Create a folder “convo” within the “spec” folder. Add an Excel file to define the conversations you want the chatbot to be able to handle — here is a sample for our IBM Watson chatbot.
The Excel is a visual representation of a conversation: on the left side there is the text the user says, and on the right side there is the text the Bot is expected to say (simple pattern matching is used).
Note: you can add several test conversations on a single Excel sheet, just add one empty row between them. You can also add additional sheets to the Excel file, or even add several Excel files to the convo-folder.
Step 5: Run the test cases
Now there is the moment of truth: right click on “package.json”, open “Run As” / “Run Configurations”, create a new “NPM” configuration with Command set to “test”.
The test conversations defined in the Excel file are executed against the chatbot and the output shows the test report.
NOTE: I published the completed workspace here. You just have to run “npm install” and “npm test”.
Looking for contributors
Please take part in the Botium community to bring chatbots forward! By contributing you help in increasing the quality of chatbots worldwide, leading to increasing end-user acceptance, which again will bring your own chatbot forward! Start here:
