Nvidia Car Learns To Drive By Watching Humans

Data Products: 9 Best Practices To Minimize Risk

Nvidia researchers have created a convolutional neural network which has learned the rules of the road by observing video from cars operated by human drivers.

The achievement demonstrates one reason there's so much excitement about artificial intelligence in the tech industry at the moment: When AI systems can learn, they require less programming.

Danny Shapiro, Nvidia's senior director of automotive, said in a blog post that the company wanted to create a robust system for driving on public roads without the need to write code to handle every specific condition a driver might encounter, a requirement he considers impractical.

Nvidia's nine-month-old project, dubbed DAVE2, represents an effort to expand on DARPA Autonomous Vehicle (DAVE) research, which aimed to create a system that could navigate public roads. DAVE ran from September 2003 through June 2004. Its success, given the standards of the time, led to further advancements in robotic navigation.

Using an Nvidia DevBox (a workstation with GPUs optimized for machine learning), Torch 7 (a machine-learning library), and an Nvidia DRIVE PX self-driving car computer, company researchers mapped the pixels of videos from a car's front-facing camera to steering commands.

"With minimum training data from humans, the system learns to drive in traffic on local roads with or without lane markings and on highways," Nvidia researchers explained in a paper describing the project. "It also operates in areas with unclear visual guidance, such as in parking lots and on unpaved roads."

The Nvidia team presented DAVE2 with about 72 hours of training data captured while a human driver traversed a variety of roads, mainly in New Jersey, but also in Illinois, Michigan, New York, and Pennsylvania, during a range of weather and lighting conditions.

After testing DAVE2 in a simulator, the team took it for a spin on public roads.

The original DAVE was a poor driver by today's standards. According to Nvidia's researchers, it could typically travel about 20 meters in complex environments before crashing. DAVE2 does a lot better.

"For a typical drive in Monmouth County, N.J., from our office in Holmdel to Atlantic Highlands, we are autonomous approximately 98% of the time," the researchers said. "We also drove 10 miles on the Garden State Parkway (a multi-lane divided highway with on- and off-ramps) with zero intercepts."

Advancements in deep learning have also taught Nvidia to see itself in a different light. The unexpected utility of GPU-powered number crunching for AI applications has helped Nvidia look beyond graphics and gaming applications toward a more central role in the computing industry.

As Yann LeCun, director of AI research for Facebook, put it last year, "It's amazing to me that so many leading companies, such as Nvidia, are changing their strategy because of deep learning and convolutional [neural] nets!"

Earlier this year, Jen-Hsun Huang, co-founder and CEO of Nvidia, described deep learning as  "a fundamentally new software model" and said his company is "betting that GPU-accelerated computing is the horse to ride."

Better still, it could be the self-driving car to ride.