Cloud-based Hadoop solution helps pharmaceutical company Astellas compress pieces of its market research from weeks to hours.
Top 10 Cloud Fiascos
(click image for larger view)
In the high-stakes game of marketing a new drug, understanding what's working and what's not can make the difference between success and expensive failure.
For top-20 pharmaceutical company Astellas, a new product launch last year presented the marketing analytics team with a data-velocity problem: Data from the field was arriving much faster than before.
"We started getting weekly patient data, instead of monthly," Chad Dau, associate director of marketing analytics at Astellas told InformationWeek by phone. The accelerated pace meant the team couldn't properly process and report on the data before the next batch arrived.
Adding complexity, Astellas and other drug companies increasingly are incorporating new data sets into their analyses, including unstructured text, audio, and video.
Dau turned to NewVantage, a Boston-based data and analytics consultancy, to pilot a cloud-based Hadoop application, leveraging its expertise in big data and optimized Hadoop configurations.
"We went from a week to three hours," said Dau, adding that Astellas plans to expand its cloud Hadoop approach next year.
"Their in-house solutions took a long time to run, even though they had a data warehouse and analysis tools," NewVantage managing partner Paul Barth told InformationWeek. For instance, in one case, analyzing how well patients take their medication -- called "patient persistence" in medical circles -- the NewVantage configuration was able to process the data 100 times faster.
"Their older style took a week or so to complete," Barth said. "We did it in under an hour."
Nor is that kind of performance boost atypical, Barth said.
"In our experience with a dozen of these set-ups, that speed-up is pretty typical," he said, noting that in some cases simply translating the data processing logic from mainframe SQL to Hadoop results in that kind of speed increase.
Paralleling these performance gains are the speed of these deployments, Barth said. "Unlike data warehouses and SQL that took a year or two to figure out if they worked, these [solutions] tell you if they're valuable in six weeks," he said.
But Barth doesn't think this inevitably drives every customer to real-time analytics.
"Some business processes are naturally slowly changing," he said. For these businesses, it wouldn't make sense to change go-to-market strategies on a daily basis. In other scenarios, such as fraud detection, time is of the essence, of course.
A Gartner survey found 64% were investing or planning to invest in big data technology in 2013. In the June survey of 720 of its worldwide clients, Gartner found 55% are currently addressing enhanced customer experience using big data, while 49% are using big data to address process efficiency.
On the other hand, Barth said there does seem to be a trend toward capturing the raw data and keeping it in a place where ad-hoc analysis is possible, such as a stock trading application that can access a repository with 10 years of transaction history in it.
"This allows you to see trends, patterns you hadn't seen before," he said.
Ellis Booker, based in Evanston, Ill., covers big data, data analytics, and education technology for InformationWeek. A familiar name in the computer trade press, he has held senior editorial posts at top IT publications including InternetWeek, Web Week, and IDG's Computerworld.
You can use distributed databases without putting your company's crown jewels at risk. Here's how. Also in the Data Scatter issue of InformationWeek: A wild-card team member with a different skill set can help provide an outside perspective that might turn big data into business innovation. (Free registration required.)
6 Tools to Protect Big DataMost IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.
Big Data Brings Big Security ProblemsWhy should big data be more difficult to secure? In a word, variety. But the business won’t wait to use it to predict customer behavior, find correlations across disparate data sources, predict fraud or financial risk, and more.
InformationWeek Tech Digest, Nov. 10, 2014Just 30% of respondents to our new survey say their companies are very or extremely effective at identifying critical data and analyzing it to make decisions, down from 42% in 2013. What gives?