“To be honest, I’m a little worried about the bot hype overtaking the bot reality,” said M.G. Siegler, a partner with GV, the investment firm formerly known as Google Ventures. “Yes, the high level promise of what bots can offer is great. But this isn’t going to happen overnight. And it’s going to take a lot of experimentation and likely bot failure before we get there.”
This reference architecture describes how to build an enterprise-grade conversational bot (chatbot) using the Azure Bot Framework. Each bot is different, but there are some common patterns, workflows, and technologies to be aware of. Especially for a bot to serve enterprise workloads, there are many design considerations beyond just the core functionality. This article covers the most essential design aspects, and introduces the tools needed to build a robust, secure, and actively learning bot.
The chatbot must rely on spoken or written communications to discover what the shopper or user wants and is limited to the messaging platform’s capabilities when it comes to responding to the shopper or user. This requires a much better understanding of natural language and intent. It also means that developers must write connections to several different platforms, again like Messenger or Slack, if the chatbot is to have the same potential reach as a website.
Once your bot is running in production, you will need a DevOps team to keep it that way. Continually monitor the system to ensure the bot operates at peak performance. Use the logs sent to Application Insights or Cosmos DB to create monitoring dashboards, either using Application Insights itself, Power BI, or a custom web app dashboard. Send alerts to the DevOps team if critical errors occur or performance falls below an acceptable threshold.
Simplified and scripted. Chatbot technology is being tacked on to the broader AI message, and while it’s important to note that machine learning will help chatbots get better at understand and responding to questions, it’s not going to make them the conversationalists we dream them to be. No matter what the marketing says, chatbots are entirely scripted. User says x, chatbot responds y.
Amazon’s Echo device has been a surprise hit, reaching over 3M units sold in less than 18 months. Although part of this success can be attributed to the massive awareness-building power of the Amazon.com homepage, the device receives positive reviews from customers and experts alike, and has even prompted Google to develop its own version of the same device, Google Home.
Chatbots can reply instantly to any questions. The waiting time is ‘virtually’ 0 (see what I did there?). Even if a real person eventually shows up to fix the issues, the customer gets engaged in the conversation, which can help you build trust. The problem could be better diagnosed, and the chatbot could perform some routine checks with the user. This saves up time for both the customer and the support agent. That’s a lot better than just recklessly waiting for a representative to arrive.
User message. Once authenticated, the user sends a message to the bot. The bot reads the message and routes it to a natural language understanding service such as LUIS. This step gets the intents (what the user wants to do) and entities (what things the user is interested in). The bot then builds a query that it passes to a service that serves information, such as Azure Search for document retrieval, QnA Maker for FAQs, or a custom knowledge base. The bot uses these results to construct a response. To give the best result for a given query, the bot might make several back-and-forth calls to these remote services.
There is no one right answer to this question, as the best solution will depend upon the specifics of your scenario and how the user would reasonably expect the bot to respond. However, as your conversation complexity increases dialogs become harder to manage. For complex branchings situations, it may be easier to create your own flow of control logic to keep track of your user's conversation.
“Beware though, bots have the illusion of simplicity on the front end but there are many hurdles to overcome to create a great experience. So much work to be done. Analytics, flow optimization, keeping up with ever changing platforms that have no standard. For deeper integrations and real commerce like Assist powers, you have error checking, integrations to APIs, routing and escalation to live human support, understanding NLP, no back buttons, no home button, etc etc. We have to unlearn everything we learned the past 20 years to create an amazing experience in this new browser.” — Shane Mac, CEO of Assist
“The chat space is sort of the last unpolluted space [on your phone],” said Sam Mandel, who works at the startup studio Betaworks and is also building a weather bot for Slack called Poncho. “It’s like the National Park of people’s online experience. Right now, the way people use chat services, it’s really a good private space that you control.” (That, of course, could quickly go sour if early implementations are too spammy or useless.)
Your bot can use other AI services to further enrich the user experience. The Cognitive Services suite of pre-built AI services (which includes LUIS and QnA Maker) has services for vision, speech, language, search, and location. You can quickly add functionality such as language translation, spell checking, sentiment analysis, OCR, location awareness, and content moderation. These services can be wired up as middleware modules in your bot to interact more naturally and intelligently with the user.
In a particularly alarming example of unexpected consequences, the bots soon began to devise their own language – in a sense. After being online for a short time, researchers discovered that their bots had begun to deviate significantly from pre-programmed conversational pathways and were responding to users (and each other) in an increasingly strange way, ultimately creating their own language without any human input.
As IBM elaborates: “The front-end app you develop will interact with an AI application. That AI application — usually a hosted service — is the component that interprets user data, directs the flow of the conversation and gathers the information needed for responses. You can then implement the business logic and any other components needed to enable conversations and deliver results.”
As in the prior method, each class is given with some number of example sentences. Once again each sentence is broken down by word (stemmed) and each word becomes an input for the neural network. The synaptic weights are then calculated by iterating through the training data thousands of times, each time adjusting the weights slightly to greater accuracy. By recalculating back across multiple layers (“back-propagation”) the weights of all synapses are calibrated while the results are compared to the training data output. These weights are like a ‘strength’ measure, in a neuron the synaptic weight is what causes something to be more memorable than not. You remember a thing more because you’ve seen it more times: each time the ‘weight’ increases slightly.
The bot (which also offers users the opportunity to chat with your friendly neighborhood Spiderman) isn’t a true conversational agent, in the sense that the bot’s responses are currently a little limited; this isn’t a truly “freestyle” chatbot. For example, in the conversation above, the bot didn’t recognize the reply as a valid response – kind of a bummer if you’re hoping for an immersive experience.
Indeed, this is one of the key benefits of chatbots – providing a 24/7/365 presence that can give prospects and customers access to information no matter when they need it. This, in turn, can result in cost-savings for companies that deploy chatbots, as they cut down on the labour-hours that would be required for staff to manage a direct messaging service every hour of the week.
Malicious chatbots are frequently used to fill chat rooms with spam and advertisements, by mimicking human behavior and conversations or to entice people into revealing personal information, such as bank account numbers. They are commonly found on Yahoo! Messenger, Windows Live Messenger, AOL Instant Messenger and other instant messaging protocols. There has also been a published report of a chatbot used in a fake personal ad on a dating service's website.