Big Data Planning: Start Slow, Let It Grow

Don't begin with an end in mind. Allow yourself to be surprised by what big data reveals.

Shane O'Neill, Managing Editor, InformationWeek

August 20, 2013

3 Min Read

"We didn't know in the past that strawberry Pop-Tarts increase in sales, like seven times their normal sales rate, ahead of a hurricane," said former Wal-Mart CIO Linda Dillman, quoted in the book. "And the pre-hurricane top-selling item was beer."

A more recent example of data insights is Netflix, considered a big-data pioneer for how it analyzes user data (and metadata) such as movie ratings, hours of video streamed, and geo-location and social media data to understand viewing habits. Last year the company discovered that 50,000 Netflix subscribers watched all 13 episodes of AMC hit show Breaking Bad's fourth season the day before Season 5 premiered, giving the company a new level of awareness about binge-watching and peak viewing activity.

Cultivate Big Data Expertise Internally And Spread The Word

There's a shortage of big data expertise, our InformationWeek Staffing Survey confirms, so companies would be wise to nurture that expertise internally.

"You already have talent within your organization -- you just don’t know it," writes InformationWeek contributor and president of Yeoman Technology Group Mike Healey in a November 2012 InformationWeek Research report titled "6 Big Data Lies." "Consider that 39% of respondent organizations have department-level analysts as the primary users of their information. Break these people out of their department silos and start moving them toward a more holistic view of the data. You’ll be amazed at how compartmentized their analysis has been."

Big data training for technical and non-technical employees is on the rise. Storage giant EMC, for instance, offers a data science and big data analytics training and certification program to both employees and customers.

"Having a few specialists isn't going to cut it," writes Simon. "Everyday employees will need to become more comfortable working with data. Much like communication skills, proficiency with data and analytics will become requisite skills."

Nevertheless, cultivating data scientists internally is time-consuming, and many companies would rather buy data scientists than cultivate them. If you go outside, snap up people quickly. Big data experts don't stay on the labor market very long, so Simon recommends that companies "aggressively target graduating and newly minted data scientists."

As you spread the big data mantra around your company, remember that not everyone will listen. But pay the naysayers no heed, writes Simon.

"As your organization gets its arms around big data and starts to see the benefits, it's likely that others will follow," he says. "Part and parcel to this is publicizing the use of new tools, data and mindsets on company intranets and social networks."

The big data market is not just about technologies and platforms -- it's about creating new opportunities and solving problems. The Big Data Conference provides three days of comprehensive content for business and technology professionals seeking to capitalize on the boom in data volume, variety and velocity. The Big Data Conference happens in Chicago, Oct. 21-23.

About the Author(s)

Shane O'Neill

Managing Editor, InformationWeek

Shane O'Neill is Managing Editor for InformationWeek. Prior to joining InformationWeek, he served in various roles at, most notably as assistant managing editor and senior writer covering Microsoft. He has also been an editor and writer at eWeek and TechTarget. Shane's writing garnered an ASBPE Bronze Award in 2011 for his blog, "Eye on Microsoft", and he received an honorable mention at the 2010 min Editorial & Design Awards for "Online Single Article." Shane is a graduate of Providence College and he resides in Boston.

Never Miss a Beat: Get a snapshot of the issues affecting the IT industry straight to your inbox.

You May Also Like

More Insights