Big Data Apps: The Next Big Thing?

What's the best solution to the looming shortage of data scientists, those high priests of analytics who glean meaning from big data?

One option is to build big data applications that automate many data scientist tasks, thereby enabling less technical business workers to make data-driven decisions without first consulting the resident data guru.

In a similar vein, big data can play a major role in the development of learning machines that make recommendations, not simply serve up results and leave the analysis and interpretation up to humans.

In a phone interview with InformationWeek, Opera Solutions chief strategy officer Laura Teller predicted that a growing sophistication in software and machine learning will help enterprises cope with the rising velocity, variety and volume of data in the coming years.

[ Big data has value that's often not reflected in the books. Read more at What's Your Big Data Worth? ]

Opera Solutions is a predictive analytics firm that employs more than 230 data scientists -- nearly a third of its staff. In 2012 it partnered with Oracle and SAP to connect the Oracle Exadata and SAP HANA data appliances with Opera Solutions' Signal Hub technologies, which use machine learning and data science to pull domain- and business-problem information from big data flows.

"The human brain was not meant to deal with this massive flood of information," Teller told InformationWeek. "And a machine has to stand between that flood of information and humans' ability to interpret and take action based on the information."

A new generation of learning machines needs to distill core information from the noise of big data and present it in ways that allow humans to take action. "We spend a lot of time thinking about this with our interfaces," Teller said. "We want the machine to serve up a set of directed actions in every application that we create. We want the machine to make recommendations to humans about what you should do, what you can do."

The development of big data applications is an emerging trend that Opera Solutions predicts could grow significantly in 2013. "If you can prepackage the science into something that's prebuilt, you can insert it on top of existing systems and workflows, and push it into the world of the operator," Teller explained.

For instance, healthcare is one industry that could benefit from prepackaged big data apps. "The area of healthcare billing, particularly hospital billing, is fraught with errors," Teller said. "A lot of it is handwritten and happens very quickly. So hospitals miss a ton of bills that they could -- and should -- legitimately bill for."

Hospitals today often use rules-based systems for billing. For instance, if one medical procedure appears on a bill, then an associated required procedure should be listed too. But Opera Solutions suggests an alternative: a patterns-based approach that studies how patients, diagnoses, and hospitals "behave" in the billing process. "We can find -- with much greater accuracy -- things that have been potentially dropped off the bill and serve those up to humans," said Teller. "We lay this on top of their existing system. It takes us about 500 man-hours to be able to hook in and train the models, which really isn't very much when you think about how much money is at stake here."

Another benefit of a patterns-based app is that it can continue to learn without human intervention. "You don't have to stop and reprogram it -- like you have to do for a rules-based system every time the rules change," Teller pointed out.

Another emerging trend to watch: The linking of a company's valuation with its big data stockpile. "I think there's going to be (more) people who help investors value companies on the basis of big data equity," said Teller. She predicts that a new "science and art" of valuing a company based on the data it has, the data it can attract, and what it can do with that data is going to come to the forefront. "And it's going to be as important as brand equity."

Cloud computing, virtualization and the mobile explosion create computing demands that today’s servers may not meet. Join Dell executives to get an in-depth look at how next-generation servers meet the evolving demands of enterprise computing, while adapting to the next wave of IT challenges. Register for this Dell-sponsored webcast now.