Google Developing Panic Button To Kill Rogue AI - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Data Management // IoT

Google Developing Panic Button To Kill Rogue AI

As Google develops artificial intelligence that has smarter-than-human capabilities, it's teamed up with Oxford University researchers to create a panic button to interrupt a potentially rogue AI agent.

Google I/O 2016: AI, VR Get Day In The Sun
Google I/O 2016: AI, VR Get Day In The Sun
(Click image for larger view and slideshow.)

With artificial intelligence crossing milestones in its capability to learn rapidly from its environment and beat humans at tasks and games from Jeopardy to the ancient Chinese game Go, Alphabet's Google is taking proactive steps to ensure that the technology it is creating does not one day turn against humans.

Google's AI research lab in London, DeepMind, teamed up with Oxford University's Future of Humanity Institute to explore ways to prevent an AI agent from going rogue. In their joint-study, "Safely Interruptible Agents," the DeepMind-Future of Humanity team proposed a framework to allow humans to repeatedly and safely interrupt an AI agent's reinforcement learning.

But, more importantly, this can be done while simultaneously blocking an AI agent's ability to learn how to prevent a human operator from turning off its machine-learning capabilities or reinforcement learning.

(Image: Henrik5000/iStockphoto)

(Image: Henrik5000/iStockphoto)

It's not a stretch to think AI agents can learn how to outthink humans. Earlier this year, Google's AI agent AlphaGo beat world champion Lee Sedol in Go, the ancient Chinese game of strategy.

By beating Lee, AlphaGo demonstrated the potential that an AI agent has for learning from its mistakes and discovering new strategies -- a characteristic that humans have.

In the joint study, the researchers looked at AI agents working in real-time with human operators. It considered scenarios when the human operators would need to press a big red button to prevent the AI agent continuing with actions that either harmed it, its human operator, or the environment around it, and teach or lead the AI agent to a safer situation.

"However, if the learning agent expects to receive rewards from this sequence, it may learn in the long run to avoid such interruptions, for example by disabling the red button -- which is an undesirable outcome," the study noted.

In essence, the AI agent learns that the button is like a coveted piece of candy. The agent wants to ensure it always has access to that button, and that any entities that could block its access, aka human operators, should be removed from the equation. That was one of the concerns expressed by Daniel Dewey, a Future of Humanity Institute research fellow, in an interview with publication Aeon in 2013.

This thinking was not lost on Google's DeepMind team, which developed AlphaGo. When Google acquired the AI company in 2014, DeepMind founders imposed a buyout condition that Google would create an AI ethics board to follow advances that Google would make in the AI landscape, according to a Business Insider report.   

[Read AI, Machine Learning Rising in the Enterprise.]

The Future of Humanity Institute, according to Business Insider, is headed up by Nick Bostrom, who said he foresees a day within the next 100 years when AI agents will outsmart humans.

In its framework paper, Google and the Institute said:

Safe interruptibility can be useful to take control of a robot that is misbehaving and may lead to irreversible consequences, or to take it out of a delicate situation, or even to temporarily use it to achieve a task it did not learn to perform or would not normally receive rewards for [...].

We have shown that some algorithms like Q-learning are already safely interruptible, and some others like Sarsa are not, off-the-shelf, but can easily be modified to have this property. We have also shown that even an ideal agent that tends to the optimal behaviour in any (deterministic) computable environment can be made safely interruptible. However, it is unclear if all algorithms can be easily made safely interruptible.

The researchers also raised a question regarding the interruption probability:

One important future prospect is to consider scheduled interruptions, where the agent is either interrupted every night at 2 am for one hour, or is given notice in advance that an interruption will happen at a precise time for a specified period of time. For these types of interruptions, not only do we want the agent to not resist being interrupted, but this time we also want the agent to take measures regarding its current tasks so that the scheduled interruption has minimal negative effect on them. This may require a completely different solution.

The need and desire to teach these AI agents how not to learn may seem counterintuitive on the surface, but could potentially keep humankind out of harm's way.

Dawn Kawamoto is an Associate Editor for Dark Reading, where she covers cybersecurity news and trends. She is an award-winning journalist who has written and edited technology, management, leadership, career, finance, and innovation stories for such publications as CNET's ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
moarsauce123
50%
50%
moarsauce123,
User Rank: Ninja
6/10/2016 | 7:20:12 AM
Re: It's Got To Be Physical
Agreed...although a wrecking ball or big axe probably work as well.
Michelle
50%
50%
Michelle,
User Rank: Ninja
6/8/2016 | 2:33:12 PM
Re: It's Got To Be Physical
This oft-used sci-fi strategy is looking better all the time...
jimbo0117
50%
50%
jimbo0117,
User Rank: Strategist
6/8/2016 | 12:05:35 PM
It's Got To Be Physical
Software can ALWAYS be circumvented, the only true fail-safe with AI will be physical circuit breakers to cut the power.
vnewman2
50%
50%
vnewman2,
User Rank: Ninja
6/6/2016 | 5:11:47 PM
Re: The Google "Kill Switch" for AI
@jastroff - I have the Tide button among others.  My 3 year old son thinks they are toys - they should have a warning;  Keep out of reach of children!
jastroff
50%
50%
jastroff,
User Rank: Ninja
6/6/2016 | 4:52:09 PM
Re: The Google "Kill Switch" for AI
Maybe we need a Panic Button for Amazon's Alexa - when it start delivering boxes of Tide enough to fill up every room.

A panic button is something Amazon would never think of. Two very different companies.
vnewman2
100%
0%
vnewman2,
User Rank: Ninja
6/6/2016 | 8:17:46 AM
Re: The Google "Kill Switch" for AI
Well it looks like they've done the math to prove it can be done. I'll be curious to see how it translates into reality.
GregG510
100%
0%
GregG510,
User Rank: Apprentice
6/4/2016 | 6:16:00 PM
The Google "Kill Switch" for AI
It is about time that the means to stop out of control AI is found.

I was thinking something simple, like a tape-loop of William Shatner arguing with the AI.

If television has taught me anything, it was the Captain Kirk could talk any computer to death.
News
Rethinking IT: Tech Investments that Drive Business Growth
Jessica Davis, Senior Editor, Enterprise Apps,  10/3/2019
Slideshows
IT Careers: 12 Job Skills in Demand for 2020
Cynthia Harvey, Freelance Journalist, InformationWeek,  10/1/2019
Commentary
Six Inevitable Technologies and the Milestones They Unlock
Guest Commentary, Guest Commentary,  10/3/2019
White Papers
Register for InformationWeek Newsletters
State of the Cloud
State of the Cloud
Cloud has drastically changed how IT organizations consume and deploy services in the digital age. This research report will delve into public, private and hybrid cloud adoption trends, with a special focus on infrastructure as a service and its role in the enterprise. Find out the challenges organizations are experiencing, and the technologies and strategies they are using to manage and mitigate those challenges today.
Video
Current Issue
Data Science and AI in the Fast Lane
This IT Trend Report will help you gain insight into how quickly and dramatically data science is influencing how enterprises are managed and where they will derive business success. Read the report today!
Slideshows
Flash Poll