Bots will revolutionize the way we search
Let’s see how bots will change the way we search and find information.
Please watch this video to see bot-powered search:
Using the power of natural language processing and Intellogo’s cognitive platform we’re inventing new, more productive ways of doing search.
Fetch is a search and recommendation bot running on Slack. In the example shown in the video, Fetch is helping out in a legal channel. When answering legal queries, Fetch has access to domain specific sources (like court docs) and understands concepts specific to law (like the difference between Supreme Court rulings and regular court docs). Since Fetch is powered by Intellogo’s cognitive platform it understands concepts like “analytical”, “liberal point of view” and “persuasive” — things which cannot be captured by keywords. It also understands how much these concepts are present in articles, books, videos, movies, etc.
Highlights of what you’re seeing:
- Search feels natural as a conversation
- A conversation helps focus the search
- The goal is to return only a single result — not pages and pages
- There’s feedback from the bot telling you what and why it's returning a certain result
- Through context management on the bot’s end, the user can easily change filters
- The bot applies its understanding of concepts and uses its own judgement to find exactly the right results
- If the bot needs help, it asks questions
What’s happening behind the scenes:
- We’re doing sophisticated natural language processing (NLP) to try and understand the intent of the user (find entertaining videos, get news or in this case, search legal docs)
- Once the bot understands the intent, it identifies the appropriate cognitive actions that it needs to perform in order to carry out the intent (analyze, focus, judge, learn)
- The bot then applies the appropriate cognitive actions against its huge corpus of content (in this case it knows to focus on new articles, seeking out analytical commentaries about same sex marriage)
- As recommendations are returned to the user, the bot maintains context of the conversation as well as its understanding of the user to provide helpful feedback about results (e.g. this recommendation may not be as analytical as you like)
- The bot adjusts its context based on what the user is saying (in this case, it tries to seek out recommendations which are more analytical than the previous recommendation)
- The bot proactively recommends removing or adding constraints in order to get different results which may interest the user (in this case, removing analytical)
Conversation will change the way we think about search. Its natural back and forth process and can be powerful when done correctly. But to make it work well the bot has to have cognitive abilities as well as NLP. When properly trained, the results are spectacular and allow us to focus on exactly the right results versus pages of results.
Here’s the transcript of the conversation with the author’s notes:
Neil: Fetch, I need an analysis of the recent supreme court ruling on same sex marriage
When the conversation starts, Fetch bot has to determine the intent of the user. In this case, it determines that we're doing a legal query based on “supreme court ruling”. It begins to break down the query into meaningful parts using natural language processing (NLP). Those parts are highlighted in bold. At this point, the bot already knows that it will look for analysis (a kind of format generally found in news) about same sex marriage.
Fetch: Shall I search for actual court cases?
The bot is coming back with an additional clarification question because the initial query specified a legal query but also a news analysis. It wants to know if it should focus on the legal aspect and search only legal sources. If the user had answered “Yes” the bot would focus its search on court documents and legal briefs.
Neil: No. I want news or commentary that cites the ruling
The user clarifies that they are looking for news and specifically commentary. Commentary is another format for news. The bot has to be able to differentiate between different news formats like: interviews, reviews, commentaries, analysis. Formats can be combined so you could have an interview that is also an analysis. The bots is able to combine these concepts so in this case it combines commentary with analysis.
Fetch: Got it. Any other focus?
The purpose of the bot is to return fewer results which are more relevant. At any given time it knows how many results are going to be returned based on the current state of the conversation. In this case it asks a final question to help winnow down the results.
Neil: Yes, focus on children
The bot now has everything that it needs and sends the intent along with everything else it has just learned to its cognitive processing center for fulfillment. The purpose of NLP is to syntactically understand what the user is asking. The purpose of cognitive processing is to semantically understand it. The cognitive center gives meaning to the query and all the content in its corpus and uses its own judgement combined with its understanding of the user to fulfill the request.
Fetch: Here’s a somewhat analytical commentary about gay marriage and children as it relates to the supreme court ruling. It takes a liberal point of view but it’s pretty persuasive.
The cognitive center has processed the request and sends back its recommendation along with a description of its reasoning to the user. This is important because it builds a trust relationship between the bot and the user. Here the bot tells the user that it was able to fulfill the request but that the most relevant result is not necessarily the most analytical. In this example, the cognitive center had access to my profile and knows (based on what I've read and requested from the bot previously) that on this particular topic a liberal view will appeal to me.
It's important to point out the power of what's happening here. The cognitive center must understand what “analytical” and “a liberal viewpoint” mean. It has to be able to recognize those things in an article. It then has to use its own judgement to determine “how analytical?” or “how liberal?” is acceptable. Then it has to compare that with what it knows about me and my tolerances. And, it has to do this for hundreds of thousands of articles, books and videos.
That’s a ton of computation but it allows the bot to relate to its users on a more conversational and human level.
Neil: I’d like something a bit more analytical
This is a powerful example of a bot using context in a conversation. The user is referring to the article that was just returned. The bot has to understand that the user wants to refine the query based on the result returned. In this case, take whatever the analytical level is in the current results and dial it up. Then return the recommendation.
Fetch: I couldn’t find any more news or commentary that is more analytical but I did find a good Wikipedia article… If you remove the constraint for analytical there are some good news articles…
The cognitive center has determined that there are no more news or commentaries on the subject that have a higher level of analysis. However, it proactively returns a Wikipedia article because it is definitely about same sex marriage and has a higher level of analysis. It takes further proactive action by recommending the removal of analysis as a requirement. This will result in some articles that it knows the user will like.
Conversations need to be proactive and helpful. A lot of bots are proactive but not necessarily in a helpful way. Putting up a button for “more results” is proactive. Knowing what the user wants and then proactively returning results based on that information is helpful. We want to build bots that are both proactive and helpful.
Neil: remove it.
Fetch: Here is the best news article about gay marriage with a focus on children and related to the supreme court ruling that I could find. Keep in mind that it is not analytical.
Here are the results from removing the analytical constraint. These are all really good results.
By combining the power of NLP with cognitive processing, we are able to create more natural and fluid interfaces. Interfaces that adapt to the user and are helpful. The results speak for themselves. The user spends less time in a search engine and more time in the results. And it's all made possible by chat bots (with a little help from my company Intellogo ;-)
I’m interested in hearing if you would use a bot like Fetch? Would you find it helpful? Are you interested in building your own bot that has cognitive abilities?
