Google Developing Panic Button To Kill Rogue AI - InformationWeek
IoT
IoT
Data Management // IoT
News
6/4/2016
11:06 AM
50%
50%
RELATED EVENTS
[Dark Reading Crash Course] Finding & Fixing Application Security Vulnerabilitie
Sep 14, 2017
Hear from a top applications security expert as he discusses key practices for scanning and securi ...Read More>>

Google Developing Panic Button To Kill Rogue AI

As Google develops artificial intelligence that has smarter-than-human capabilities, it's teamed up with Oxford University researchers to create a panic button to interrupt a potentially rogue AI agent.

Google I/O 2016: AI, VR Get Day In The Sun
Google I/O 2016: AI, VR Get Day In The Sun
(Click image for larger view and slideshow.)

With artificial intelligence crossing milestones in its capability to learn rapidly from its environment and beat humans at tasks and games from Jeopardy to the ancient Chinese game Go, Alphabet's Google is taking proactive steps to ensure that the technology it is creating does not one day turn against humans.

Google's AI research lab in London, DeepMind, teamed up with Oxford University's Future of Humanity Institute to explore ways to prevent an AI agent from going rogue. In their joint-study, "Safely Interruptible Agents," the DeepMind-Future of Humanity team proposed a framework to allow humans to repeatedly and safely interrupt an AI agent's reinforcement learning.

But, more importantly, this can be done while simultaneously blocking an AI agent's ability to learn how to prevent a human operator from turning off its machine-learning capabilities or reinforcement learning.

(Image: Henrik5000/iStockphoto)

(Image: Henrik5000/iStockphoto)

It's not a stretch to think AI agents can learn how to outthink humans. Earlier this year, Google's AI agent AlphaGo beat world champion Lee Sedol in Go, the ancient Chinese game of strategy.

By beating Lee, AlphaGo demonstrated the potential that an AI agent has for learning from its mistakes and discovering new strategies -- a characteristic that humans have.

In the joint study, the researchers looked at AI agents working in real-time with human operators. It considered scenarios when the human operators would need to press a big red button to prevent the AI agent continuing with actions that either harmed it, its human operator, or the environment around it, and teach or lead the AI agent to a safer situation.

"However, if the learning agent expects to receive rewards from this sequence, it may learn in the long run to avoid such interruptions, for example by disabling the red button -- which is an undesirable outcome," the study noted.

In essence, the AI agent learns that the button is like a coveted piece of candy. The agent wants to ensure it always has access to that button, and that any entities that could block its access, aka human operators, should be removed from the equation. That was one of the concerns expressed by Daniel Dewey, a Future of Humanity Institute research fellow, in an interview with publication Aeon in 2013.

This thinking was not lost on Google's DeepMind team, which developed AlphaGo. When Google acquired the AI company in 2014, DeepMind founders imposed a buyout condition that Google would create an AI ethics board to follow advances that Google would make in the AI landscape, according to a Business Insider report.   

[Read AI, Machine Learning Rising in the Enterprise.]

The Future of Humanity Institute, according to Business Insider, is headed up by Nick Bostrom, who said he foresees a day within the next 100 years when AI agents will outsmart humans.

In its framework paper, Google and the Institute said:

Safe interruptibility can be useful to take control of a robot that is misbehaving and may lead to irreversible consequences, or to take it out of a delicate situation, or even to temporarily use it to achieve a task it did not learn to perform or would not normally receive rewards for [...].

We have shown that some algorithms like Q-learning are already safely interruptible, and some others like Sarsa are not, off-the-shelf, but can easily be modified to have this property. We have also shown that even an ideal agent that tends to the optimal behaviour in any (deterministic) computable environment can be made safely interruptible. However, it is unclear if all algorithms can be easily made safely interruptible.

The researchers also raised a question regarding the interruption probability:

One important future prospect is to consider scheduled interruptions, where the agent is either interrupted every night at 2 am for one hour, or is given notice in advance that an interruption will happen at a precise time for a specified period of time. For these types of interruptions, not only do we want the agent to not resist being interrupted, but this time we also want the agent to take measures regarding its current tasks so that the scheduled interruption has minimal negative effect on them. This may require a completely different solution.

The need and desire to teach these AI agents how not to learn may seem counterintuitive on the surface, but could potentially keep humankind out of harm's way.

Dawn Kawamoto is an Associate Editor for Dark Reading, where she covers cybersecurity news and trends. She is an award-winning journalist who has written and edited technology, management, leadership, career, finance, and innovation stories for such publications as CNET's ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
moarsauce123
50%
50%
moarsauce123,
User Rank: Ninja
6/10/2016 | 7:20:12 AM
Re: It's Got To Be Physical
Agreed...although a wrecking ball or big axe probably work as well.
Michelle
50%
50%
Michelle,
User Rank: Ninja
6/8/2016 | 2:33:12 PM
Re: It's Got To Be Physical
This oft-used sci-fi strategy is looking better all the time...
jimbo0117
50%
50%
jimbo0117,
User Rank: Strategist
6/8/2016 | 12:05:35 PM
It's Got To Be Physical
Software can ALWAYS be circumvented, the only true fail-safe with AI will be physical circuit breakers to cut the power.
vnewman2
50%
50%
vnewman2,
User Rank: Ninja
6/6/2016 | 5:11:47 PM
Re: The Google "Kill Switch" for AI
@jastroff - I have the Tide button among others.  My 3 year old son thinks they are toys - they should have a warning;  Keep out of reach of children!
jastroff
50%
50%
jastroff,
User Rank: Ninja
6/6/2016 | 4:52:09 PM
Re: The Google "Kill Switch" for AI
Maybe we need a Panic Button for Amazon's Alexa - when it start delivering boxes of Tide enough to fill up every room.

A panic button is something Amazon would never think of. Two very different companies.
vnewman2
100%
0%
vnewman2,
User Rank: Ninja
6/6/2016 | 8:17:46 AM
Re: The Google "Kill Switch" for AI
Well it looks like they've done the math to prove it can be done. I'll be curious to see how it translates into reality.
GregG510
100%
0%
GregG510,
User Rank: Apprentice
6/4/2016 | 6:16:00 PM
The Google "Kill Switch" for AI
It is about time that the means to stop out of control AI is found.

I was thinking something simple, like a tape-loop of William Shatner arguing with the AI.

If television has taught me anything, it was the Captain Kirk could talk any computer to death.
How Enterprises Are Attacking the IT Security Enterprise
How Enterprises Are Attacking the IT Security Enterprise
To learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
Register for InformationWeek Newsletters
White Papers
Current Issue
2017 State of IT Report
In today's technology-driven world, "innovation" has become a basic expectation. IT leaders are tasked with making technical magic, improving customer experience, and boosting the bottom line -- yet often without any increase to the IT budget. How are organizations striking the balance between new initiatives and cost control? Download our report to learn about the biggest challenges and how savvy IT executives are overcoming them.
Video
Slideshows
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll