Google AI Teaches Itself Atari Video Games

Using a single learning algorithm, Google's AI agent has developed its video game skills to the point where it can beat most human players in dozens of different games.
8 Google Projects To Watch in 2015
8 Google Projects To Watch in 2015
(Click image for larger view and slideshow.)

With news streaming out of Google this week that seems to show AI can learn, we ask: Where's the John Henry of the computer era?

The Industrial Age legend of John Henry is an apt analogy for our current relationship with robotics. As the story goes, Henry was a steel driver who raced a steam-powered hammer to prove that man could beat machine. According to the legend, which dates to the 1800s, Henry won the contest but his was a Pyrrhic victory. He died as a result of overexertion and machines ultimately transformed the nature of work. 

The Information Age is still looking for its John Henry.

Ken Jennings and Brad Rutter were soundly beaten at Jeopardy! by IBM's Watson in 2011. Former chess grandmaster Garry Kasparov was defeated in 1997 by Watson's precursor Deep Blue. The celebrated human brain may be able to design computers, but it can't out-compute them.

The fact is computers have been besting people at specific tasks since they were invented. And that's been for the best. But lately it has become clear that the narrow scope of machine intelligence is becoming broader.

Google has demonstrated that in research published this week in Nature. The company's London-based DeepMind team has developed an algorithm called deep Q-network (DQN) that can teach itself to play video games.

What's significant about this is that DQN relied on a single algorithm to become proficient at a variety of games through sensory input; it doesn't require a preprogrammed model of how each game works, a common approach for writing game bots. Using neural networks and a machine learning framework called Reinforcement Learning, DQN became a skilled player of dozens of Atari 2600 games.

"DQN outperformed previous machine learning methods in 43 of the 49 games," explained DeepMind researchers Dharshan Kumaran and Demis Hassabis in a blog post. "In fact, in more than half the games, it performed at more than 75% of the level of a professional human player."

Kumaran and Hassabis note that DQN even developed "surprisingly far-sighted strategies," like creating a tunnel in Breakout that allowed the ball to bounce back and forth against the back wall and the bricks.

The researchers' paper claims that DQN is "the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks."

Juergen Schmidhuber co-director of the Swiss AI Lab IDSIA in Lugano, Switzerland, and professor of artificial intelligence at the University of Lugano, said in a Google+ post that this claim is "debatable at best," noting that other learning systems can solve diverse tasks and that the first such system was developed at IDSIA -- where three of the DeepMind researchers once worked.

[ Read about how Android is going to work. ]

Nevertheless, the success of Google's researchers underscores the potential of machine intelligence as a way to create value for businesses. Kumaran and Hassabis propose that Google could use this technology to perform a task as complicated as planning a backpacking trip across Europe on behalf of a user.

For knowledge workers, the results suggest that people will have to continue to raise the level of their game to compete against the creeping competence of machines. And there's no telling how long we can maintain our mental edge or what we will do if machines can do what we do, but faster and more affordably.

In a recent blog post about the potential threat of super-intelligent machines -- an increasingly popular meme among the technically savvy -- entrepreneur Sam Altman observed that that, while it's difficult to say how soon machine intelligence will surpass human intelligence, it's dangerous to assume it can't happen.

"We decry current machine intelligence as cheap tricks, but perhaps our own intelligence is just the emergent combination of a bunch of cheap tricks," said Altman.

Attend Interop Las Vegas, the leading independent technology conference and expo series designed to inspire, inform, and connect the world's IT community. In 2015, look for all new programs, networking opportunities, and classes that will help you set your organization’s IT action plan. It happens April 27 to May 1. Register with Discount Code MPOIWK for $200 off Total Access & Conference Passes.