AI in Production: A Roadmap for Success

Many enterprises have some form of artificial intelligence and machine learning in place, whether it's in the form of a pilot program, a proof-of-concept in the cloud, or even a production implementation. Even though machine learning has been around for a long time, it's still an emerging technology for many enterprises.

For those who have created a pilot or proof-of-concept, you may have cleared the first hurdle in implementing these technologies. But putting them into production, even on a limited basis, may present challenges that you have not yet considered. Putting large-scale systems in place that deliver value is an entirely new proposition.

"It's a lot more involved than most people think," said Ted Dunning, chief application architect at MapR, a big data platform company with its roots in Hadoop. "They get this feeling that it is a magic bullet, but there are a lot of logistics around it."

Dunning and Ellen Friedman, MapR principal technologist, spoke with InformationWeek in an interview about the challenges they are seeing in the field as enterprises work to put AI into production. The pair also recently authored a new book, offering advice to enterprises that are facing this challenge: AI and Analytics In Production: How to Make it Work. Of course, these two MapR executives offer MapR as the sample platform for examples, however, the book also provides more universal recommendations that could help with implementation designed to get value out of large-scale data production systems, regardless of the platform.

"We’ve found that a lot of people have jumped in and started these projects, but haven't productionalized them," Friedman told me. Indeed, she says that only 17% to 18% of Hadoop or HDSF platform projects are in production environments.

The secret to success in creating a production system lies in the organization's approach to the data itself. Friedman said that organizations need to reframe how they handle data, to take a "Data Ops" approach. But that doesn't mean that this will be exactly like DevOps. In some ways it's more challenging and involved.

For instance, unlike DevOps, where developers work through multiple iterations of code, data production must begin sooner than code development and end later. Data has a longer memory than code, Friedman and Dunning write in the book.

"Messing up your data can cause you problems much longer than messing up your code ever could," Friedman and Dunning write.

The authors also recommend that enterprises create systems that support multitenancy -- where a single instance of software on a server can serve multiple tenants -- sort of like how multiple tenants can live in a single apartment building.

"People are surprised that we say this is an important issue," said Friedman. But it enables the enterprise infrastructure to be ready for more projects. "It means you are set up for the next and next and next project," she said.

It's that kind of production environment that really enables large scale success with machine learning and AI. The authors recommend building a global data fabric, orchestrating containers with Kubernetes, using streaming architecture and microservices, and cultivating a production-ready culture.

They also lay out a series of tips and tricks to help guide projects to success. Number one, which you've undoubtedly heard in other contexts -- "Pick one thing to do first."  Sure there will be many different applications of machine learning and AI in the future for your enterprise, and many of those you haven't even thought of yet.  But starting with just one will help get the team familiar with the technology.

"As your team becomes familiar with what is feasible and with the ecosystem tools required for your specific goal, you'll be well-positioned to try other things as you see new ways in which advanced analytical systems may be useful to you," the authors write.