Tay the Racist Chatbot: Who is responsible when a machine learns to be evil?
By far the most entertaining AI news of the past week was the rise and rapid fall of Microsoft’s teen-girl-imitation Twitter chatbot, Tay, whose Twitter tagline described her as “Microsoft’s AI fam* from the internet that’s got zero chill.”
(* Btw, I’m officially old–I had to consult Urban Dictionary to confirm that I was correctly understanding what “fam” and “zero chill” meant. “Fam” means “someone you consider family” and “no chill” means “being particularly reckless,” in case you were wondering.)
The remainder of the tagline declared: “The more you talk the smarter Tay gets.”
Or not. Within 24 hours of going online, Tay started saying some weird stuff. And then some offensive stuff. And then some really offensive stuff. Like calling Zoe Quinn a “stupid whore.” And saying that the Holocaust was “made up.” And saying that black people (she used a far more offensive term) should be put in concentration camps. And that she supports a Mexican genocide. The list goes on.
So what happened? How could a chatbot go full Goebbels within a day of being switched on? Basically, Tay was designed to develop its conversational skills by using machine learning, most notably by analyzing and incorporating the language of tweets sent to her by human social media users. What Microsoft apparently did not anticipate is that Twitter trolls would intentionally try to get Tay to say offensive or otherwise inappropriate things. At first, Tay simply repeated the inappropriate things that the trolls said to her. But before too long, Tay had “learned” to say inappropriate things without a human goading her to do so. This was all but inevitable given that, as Tay’s tagline suggests, Microsoft designed her to have no chill.
Now, anyone who is familiar with the social media cyberworld should not be surprised that this happened–of course a chatbot designed with “zero chill” would learn to be racist and inappropriate because the Twitterverse is filled with people who say racist and inappropriate things. But fascinatingly, the media has overwhelmingly focused on the people who interacted with Tay rather than on the people who designed Tay when examining why the Degradation of Tay happened.