To try to fix this, Facebook’s engineers have built their own dataset to train Facebook to teach it’s chatbot how to chit-chat with. It’s called Persona-Chat, and consists of more than 160,000 lines of dialogue, sourced from workers found on Amazon’s Mechanical Turk marketplace. (The resource for human data used to train AI.)
The clever thing about Persona-Chat is that the idle talk it contains isn’t just random. To give some structure to the data, and to address the challenge of making chatbots with personality, the Mechanical Turk workers were asked to design their own character to guide their dialogue. This meant coming up with five basic biographical statements and using them as topics of conversation. So, for example, one persona was based around the following facts: “I am an artist. I have four children. I recently got a cat. I enjoy walking for exercise. I love watching Game of Thrones.”
It’s not exactly Tolstoy, but it’s a start.
This data was used to train neural networks used for existing chatbots, with the results then assessed by another group of Mechanical Turkers. In each case, they were asked to conduct a conversation with the persona-driven bot, and compare it with both other chatbots and humans. As you might expect, the persona bot didn’t score as highly on criteria like “fluency” and “consistency” as the humans, but it outperformed the chatbot trained on movie dialogue.
Here’s a sample of conversation. Person 1 is a human, and Person 2 is the persona-based chatbot. (Its main interest, if you can’t guess, is “writing romance novels.”)
[PERSON 1:] I just got done playing soccer. What have you been up to?
[PERSON 2:] I do not have pets. I spend my free time writing. what about you?
[PERSON 1:] Just graduated from college. Looking for work.
[PERSON 2:] yes I graduated, in writing and poetry
[PERSON 1:] Have you had anything published?
[PERSON 2:] I mainly write fiction novels.
[PERSON 1:] Do you like to watch sports?
[PERSON 2:] do you like kings of leon my favorite by them is use somebody
[PERSON 1:] Are you married? I will be married soon.
[PERSON 2:] haha, no time. I have got a novel to finish
As you’d expect, it’s still not perfect small talk, but it is at least consistent. (And who hasn’t ended a conversation with the excuse “I have got a novel to finish”?) The research also points to a truth that seems almost common sense. If you want to have an interesting conversation with someone — even a virtual someone — then it helps if they have a personality, including likes and interests.
As virtual assistants like Siri and Alexa become more integrated into our lives, we can expect their creators to build out their characters. The problem then becomes whether or not we’ll get along.