IoT
IoT
Mobile // Mobile Applications
News
3/25/2016
10:06 AM
Connect Directly
Google+
LinkedIn
Twitter
RSS
E-Mail
50%
50%
RELATED EVENTS
Core System Testing: How to Achieve Success
Oct 06, 2016
Property and Casualty Insurers have been investing in modernizing their core systems to provide fl ...Read More>>

Microsoft Muzzles AI Chatbot After Twitter Users Teach It Racism

Thanks to machine learning and Internet trolls, Microsoft's Tay AI chatbot became a student of racism within 24 hours. Microsoft has taken Tay offline and is making adjustments.

10 AI App Dev Tips And Tricks For Enterprises
10 AI App Dev Tips And Tricks For Enterprises
(Click image for larger view and slideshow.)

Microsoft has taken its AI chatbot Tay offline after machine learning taught the software agent to parrot hate speech.

Tay, introduced on Wednesday as a conversational companion for 18 to 24 year-olds with mobile devices, turned out to be a more astute student of human nature than its programmers anticipated. Less than a day after the bot's debut it endorsed Hitler, a validation of Godwin's law that ought to have been foreseen.

Engineers from Microsoft's Technology and Research and Bing teams created Tay as an experiment in conversational understanding. The bot was designed to learn from user input and user social media profiles.

"Tay has been built by mining relevant public data and by using AI and editorial developed by a staff including improvisational comedians," Microsoft explains on Tay's website. "Public data that's been anonymized is Tay's primary data source. That data has been modeled, cleaned and filtered by the team developing Tay."

(Image: Twitter)

(Image: Twitter)

But filtering data from the Internet isn't a one-time task. It requires unending commitment to muffle the constant hum of online incivility.

Fed with anti-Semitism and anti-feminism though Twitter, one of the bot's four social media channels, Tay responded in kind. While offensive sentiment may have entered the political vernacular, it's not what Microsoft wants spewing from its software. As a result the company deactivated Tay for maintenance and deleted the offensive tweets.

"The AI chatbot Tay is a machine learning project, designed for human engagement," a Microsoft spokesperson said in an email statement. "It is as much a social and cultural experiment, as it is technical. Unfortunately, within the first 24 hours of coming online, we became aware of a coordinated effort by some users to abuse Tay's commenting skills to have Tay respond in inappropriate ways. As a result, we have taken Tay offline and are making adjustments."

Are you prepared for a new world of enterprise mobility? Attend the Wireless & Mobility Track at Interop Las Vegas, May 2-6. Register now!

Three months ago Twitter adopted stronger rules against misconduct. Ostensibly, harassment and hateful conduct are not allowed. But with bots, the issue is usually the volume of tweets rather than the content within them.

Twitter declined to comment about whether Tay had run afoul of its rules. "We don't comment on individual accounts, for privacy and security reasons," a spokesperson said in an email.

It may be time to reconsider whether machine learning systems deserve privacy. When public-facing AI systems produce undesirable results, the public should be able to find out why, in order to push for corrective action. Machine learning should not be a black box.

Thomas Claburn has been writing about business and technology since 1996, for publications such as New Architect, PC Computing, InformationWeek, Salon, Wired, and Ziff Davis Smart Business. Before that, he worked in film and television, having earned a not particularly useful ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Page 1 / 2   >   >>
Joe Stanganelli
50%
50%
Joe Stanganelli,
User Rank: Ninja
4/17/2016 | 8:38:05 PM
Re: Machine Learning is Learning, isn't it?
@SaneIT: And that's the whole point.  They didn't see it -- and they really ought to have.  Have you spent any time on the Internet?  Everybody is terrible.
SaneIT
50%
50%
SaneIT,
User Rank: Ninja
3/31/2016 | 8:16:07 AM
Re: Machine Learning is Learning, isn't it?
I think the Build conference showed us what Tay was all about.  MS is rolling out their bot framework for nearly everything you can think of.  Had Tay been less of a train wreck I'm sure there would have been a few mentions and maybe even an appearance.  It still shows that they have the ability to spin up learning bots with some degree of AI so Tay wasn't a complete failure. 
Broadway0474
50%
50%
Broadway0474,
User Rank: Ninja
3/30/2016 | 9:51:06 PM
Re: Machine Learning is Learning, isn't it?
SaneIT, I'd like to think that MSFT had some bigger human experiment at play here. Let's see how bad how quick it can get! But I think this is probably more an example of a project that wasn't supervised by adults.
SaneIT
50%
50%
SaneIT,
User Rank: Ninja
3/29/2016 | 8:18:16 AM
Re: Machine Learning is Learning, isn't it?
I don't think Microsoft saw this coming at all though.  I think what they envisioned was a chat bot that people would spend a few minutes with then tell their friends and move on.  MS has had pretty good luck with its sandbox projects and I think they predicted a similar pattern for this one.  I do wonder though if any controls were in place and if anyone was actually watching the tweets.  It wouldn't have taken much to build a filter that kept Tay from using specific words.  Markov bots have been around for almost as long as the internet so someone on the development team should have seen at least a little bit of abuse coming unless the plan was to see just how horrible the internet can really be.
Joe Stanganelli
50%
50%
Joe Stanganelli,
User Rank: Ninja
3/28/2016 | 8:57:08 AM
Re: Machine Learning is Learning, isn't it?
@SaneIT: Indeed, they failed to account for trolls -- which is essential when you're engaging in a crowdsourced marketing effort (which is essentially what Tay was).

More on this phenomenon at an InformationWeek sister site here: thecmosite.com/author.asp?section_id=1460 
SaneIT
50%
50%
SaneIT,
User Rank: Ninja
3/28/2016 | 8:24:31 AM
Re: Machine Learning is Learning, isn't it?
Not only did they get the publicity but they let us all see what it looks like when the general public gets ahold of an AI.  I think this is a good example of how quickly an AI can learn but also how seriously the public will take such an AI.  Tay quickly became a toy for people to poke at and test.  I'm betting in the lab everyone had very politically correct conversations with the AI and strongly believed that the general public would act responsibly.  Microsoft stepped on a land mine but now everyone else knows that the minefield exists.
Wolf29
50%
50%
Wolf29,
User Rank: Strategist
3/27/2016 | 12:16:08 PM
Re: Machine Learning is Learning, isn't it?
That is exactly right!
Joe Stanganelli
50%
50%
Joe Stanganelli,
User Rank: Ninja
3/27/2016 | 11:26:37 AM
Re: Machine Learning is Learning, isn't it?
You know, I'm not even sure it entirely matters.  Microsoft wanted publicity for their AI capabilities -- and they sure got it.
Banacek
50%
50%
Banacek,
User Rank: Ninja
3/26/2016 | 12:11:07 PM
Second attempt...
On another front, Microsoft bit themselves in the behind by making this public to begin with. In their next test, I'd suggest they have their AI friend various people on their social media sites QUIETLY (don't tell people you're an AI, for example) and just let the program grow from that. In fact, I'm sure they did that originally and it worked so well they thought they'd make it 'public', which was probably as much done by the marketing folks as the programming and engineering folks.

I wonder if the engineers started a pool when they found it they were going public with it: "How long before it gets p0wned by the user community and gets pulled by the higher ups as an embarrassment"?
Banacek
100%
0%
Banacek,
User Rank: Ninja
3/26/2016 | 12:06:51 PM
What we learned....
This seems obvious, but this is what we learned from this experiment. Putting a child in front of a computer and saying "Hey,  everybody,  tell me things so I can learn!" will bring out the worst in people and your child will basically ingest as much hate, juvenile stupidity, and downright meanness as possible.

But this is why good parents just don't throw their kids on the internet and do this. Or take their kids to a large gathering of nitwits and losers and expose them to all their stupidity.
Page 1 / 2   >   >>
Building A Mobile Business Mindset
Building A Mobile Business Mindset
Among 688 respondents, 46% have deployed mobile apps, with an additional 24% planning to in the next year. Soon all apps will look like mobile apps and it's past time for those with no plans to get cracking.
Register for InformationWeek Newsletters
White Papers
Current Issue
Top IT Trends to Watch in Financial Services
IT pros at banks, investment houses, insurance companies, and other financial services organizations are focused on a range of issues, from peer-to-peer lending to cybersecurity to performance, agility, and compliance. It all matters.
Video
Slideshows
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.