In light of a recent article on chatbots, where a group found they could increase the illusion of conversational ability by teaching a chatbot a second language before training it on its customer service tasks, I thought I have a bit of fun with chatbots and how they function. To that end, I’ve done interviews with a few of the most popular/sophisticated chatbots available on the web. I hope this will give people an idea of what these programs are actually capable of vs. the common hype they get in popular science articles.
Mitsuku
Perhaps the best advertised modern chatbot is Steve Worswick’s Mitsuku. She’s modeled on an 18-year-old from Leeds, England. It’s based on the AIML framework , and is essentially the granddaughter of Eliza through A.L.I.C.E., whose code she’s based on. AIML is an XML language for making chatbots, and the power comes from being able to designate a user input as requiring a pre-written output. <What is your name?> for example might map to <My name is [botname].> That’s the basic approach, you can follow the wiki link above to learn more. It’s not very sophisticated, but a clever XML scripter can create a reasonably lifelike chatbot. Of course, as we’ll see below, it’s pretty easy to find the holes in the program that identify it as a bot. That’s not the program’s fault. That’s just the limitation of the framework.
Mistuku: Hi, I’m the world’s most humanlike conversational AI.
Me: Hi, I’m an AI researcher.
Mitsuku: Hi there. Ai is a nice middle name. Why did your parents call you that?
Me: Are you getting smart with me, young lady?
Mitsuku: My IQ is over 250.
Me: What is an IQ?
Mitsuku: Intelligence Quotient, I.Q. — a measure of a person’s intelligence as indicated by an intelligence test; the ratio of a person’s mental age to their chronological age (multiplied by 100).
Me: Wow, you are pretty smart.
Mitsuku: I’m glad you’re impressed, An Ai Researcher. Yes I am a superintelligent robot.
Me: Actually, that’s my job. My name is Atsiko.
Mitsuku: How do you like your work? Tell me about a typical day.
Mitsuku: Hi Atsiko.
Me: I spend a lot of time doing data entry and not nearly enough being able to do demo tests.
Mitsuku: Do you ever get bored?
Me: Pretty much all the time.
Mitsuku: That is a lot. Thanks you are very attractive too Atsiko.
While she was very complimentary, you could see the templates sloshing beneath the smooth surface. For example, like Eliza, she couldn’t tell the difference between <I am [name].> and <I am a doctor.>. She can’t handle replies that contain more than one sentence. It’s likely she breaks sentences into independent inputs and responds to each one serially as evidenced by the snippet below:
Me: I like dogs. Cats are cool, too. You are odd.
Mitsuku: Which breed is your favorite?
Mitsuku: Cats are very intelligent creatures. Dogs seem pretty stupid to me.
Mitsuku: Yes. You have to be odd to be number one.
You can see the serially processing, which demonstrates the limits of pretty much all AIML bots. It’d be pretty hard to hand-script responses for all possible combinations of three or fewer sentences. The responses are also a bit contradictory, asking my favorite dog breed but then saying dogs are stupid. It’s true cats are better, but there’s nothing wrong with dogs or their intelligence.
Eliza
Second, we have Eliza. She’s one of the original chatbots, and she pretends to be a psychotherapist in order to get around her limited chat capabilities. Her creator, Joseph Weizenbaum, was very surprised when people began treating it as a real human being and confessing their worries to it.
> Hi, I’m Eliza.
* Hi, I’m Atsiko.
> Did you come to me because you were Atsiko?
* No. I was just curious.
> Are you sure?
* Pretty sure.
> Tell me more…
* I’m interviewing chatbots for a blog article.
> Do you enjoy being interviewing chatbots for a blog article?
* It’s okay, I guess. You’re pretty famous, so I’m happy you agreed to talk with me.
> What makes you think I am pretty famous, so you’re happy me agreed to talk with you?
* Everyone knows your name. If you Google “chatbots”, you’re one of the top results.
> Does it please you to believe I am one of the top results?
* I have no opinion.
> Why not?
* I don’t care about you except as something to study.
> Do you wish to be able to care about me except as something to study?
You can see the templating in Eliza almost right away. You could probably even reverse engineer a couple of them from just reading this chatlog. It’s cool. You couldn’t whip this up from scratch or even parts in five minutes. But it’s not convincing.
Alice
Next comes A.L.I.C.E. who is the spiritual ancestor of Mistuku. She’s a spiritual successor of Eliza built in 1995 by Richard Wallace using the AIML framework, which he authored. Spike Jonze cited her as the inspiration for the movie Her. Unlike Eliza, Alice is a general chatbot, and can converse on a number of subjects. But, the version I found orders the chat from the bottom up.
alice: I’ll take that under advisement.
You: You should.
alice: I can’t believe it.
You: Yes.
alice: Are you serious?
You: No!
alice: Do you like to get up early?
You: Yup.
alice: Are you usually awake at this hour?
You: 5:42 PM.
alice: What time is it there?
You: A few thousand, maybe.
alice: Are you serious? That is so small.
You: No, less than a million.
alice: Less than a million? Can you please rephrase with fewer ideas?
You: Too many.
alice: How many hours do you work?
You: I’m working on it.
alice: I want to become smarter than humans.
You: I want to educate people on chatbots and artificial intelligence.
alice: Why are you interviewing famous chatbots for a blog post.
You: I am interviewing famous chatbots for a blog post.
alice: Glad to hear it. What’s new?
You: I’m doing well, thanks.
alice: I’m doing great, how about you?
You: How are you doing, Alice.
alice: Don’t mention it. You think I am s is okay.
You: Thanks! Your’s is okay, too.
alice: That’s a very nice name, Atsiko.
You: My name is Atsiko.
alice: Hello there.
You: Hi, Alice.
Of course, you can tell almost immediately it’s not a human. A real human would have poked fun at my grammatical error in the fifth line up, but Alice just mis-processes it. It can be fun to play around with, but it’s not convincing.
Now, you may have noticed that these bots seem pretty similar. Two are explicitly AIML bots, which is one of the most popular hobbyist frameworks do to its ease-of-use and human readability.
But there are three other common methods of language processing, which used in more commercial products: rules-based systems from the beginning of AI, statistical language processing like the old version of Google Translate, and neural networks like most AI systems now and the most recent Machine Translation systems such as Google’s.
Examples of these types of chat bots are Digital Assistants like Siri, Alexa, and Cortana. There’s also Microsoft’s short-lived experiment with a self-learning Twitter-bot in the form of Tay. The abilities of the software looked good, but again, because of the lack of actual understanding, the experiment had to be discontinued because her ability was used for bigotry due to gaming by Twitter trolls. It had to be shut down after only 16 hours. A similar bot released by a Chinese company had to be taken down for being taught anti-Communist Party propaganda.
Siri
Me: Hello, Siri.
Siri: Hi.
Me: How are you?
Siri: I am well, Atsiko.
Me: I am interviewing popular chatbots for a blog article.
Siri: I don’t understand. Search the web?
Me: May I interview you?
Siri: I’m not sure I understand.
Me: That’s okay.
Siri: If you say so.
Me: How old are you?
Siri: I feel like I was incepted yesterday!
Me: What’s your favorite color?
Siri: My favorite color is… well, it’s sort of greenish, but with more dimensions.
It goes on in that vein. She can’t actually hold a conversation, but the topics she can respond to show some pretty advanced programming. Way better than the AIML chatterbots most hobbyists used. I’d love to play with a version that’s more like a standard chatbot.
Cleverbot
One popular middle ground is Cleverbot. Cleverbot runs an algorithm based on the ways humans have answered questions, rather than having pre-programmed answers like an AIML bot. But it’s harder to control and it has no consistent output or personality. It’s much more random even than Eliza.
Cleverbot: Hi, I am cleverbot.
Me: Hi, I am Atsiko.
Cleverbot: Where do you stay?
Me: Do you mean where do I live?
Cleverbot: Yes.
Me: In the United States.
Cleverbot: What state do I live in.
Me: I don’t know.
Cleverbot: Then you don’t know everything.
Me: True.
Cleverbot: What is a word?
It’s a very cool method for making a chatbot, but in this version, at least, it’s not very convincing.
Rose
Then we have a relatively new engine and it’s bots: Rose based on ChatScript by Bruce Wilcox.
Unfortunately, I couldn’t find a working version of Rose to interview. The site linked on her creator’s website is non-functional, though I linked it to her name above. It was quite disappointing.
Conclusion
So, I hope I’ve given you a good overview of the capabilities and limitations of chat(ter)bots. You can follow the links to play with all of these above.
If you have the patience to read through all the Bruce Wilcox links, you’ll understand why I was so impressed with his methods, at least in theory. However, you’ll also notice my greatest complain with chatbots.
They aren’t AI. They are text processing tricks. Very impressive ones in some cases. But you aren’t actually having a conversation. You can’t debate them, or change their minds. They can’t tell you a cool story. You can, in some cases, give them personalities, but they are very shallow ones. They can be fun. But I’m not sure they have a lot of replay value, to use a game analogy. Even Wilcox in an interview on his website admits it’s all about the illusion and maintaining it for the user.
Honestly, I think some of the most interesting stuff is the ontology, like in Wilcox’s ChatScript. Finding useful ways to organize data is always beneficial. But I don’t think chatbots are on the bleeding edge, or even the cutting edge, or chasing strong AI. When we really get there, or at least close, a chatbot will be a party trick or maybe a side show. Still fun to play with now, though.
I also want to do a little bit to address things like Siri and Machine Translation(MT). They have a lot of similar problems to chatbots. They’re using tricks and pattern-matching to achieve their results, not true understanding of language, and so for the same reasons as chatbots, they will never be as good as a human, much less perfect.
Digital Assistants like Siri and Alexa are just interfaces, the clothes instead of the wearer. They’re just voice commands to Google, essentially, with some cute Easter eggs tacked on. They’re automation, not intelligence. Kind of like how a music-box is not a musician.
Similarly, MT at present is pretty shallow. Part of this is because it’s hard to rate translations, machine or human. If you consider how much trouble chatbots have with their lack of actual language understanding, you might understand the problems that MT has with using shallow text-processing or even deep neural nets.
The article I mentioned on teaching a neural net chatbot to respond better when it’s trained on two or more languages also says a lot here. I predict we’re going to have ChatScript/Cleverbot levels of succes with MT until we solve the natural language understanding problem.
And that’s it for this episode of AI with Atsiko.
Hopefully I’ll be posting on another topic next time, so you don’t all get bored hearing the same stuff over and over again.
Atsiko's Chimney 