Big Data Holdouts Risk Getting Swamped

Emerging software tools make analytics feasible -- and cost-effective -- for most companies. Now, staying on the shore may be riskier than just diving in.
InformationWeek Green -  Sept. 30, 2013 InformationWeek Green
Download the entire InformationWeek October special issue on big data analytics, distributed in an all-digital format (registration required).

EMC says that between 2005 and 2020, total stored information, what it calls the digital universe, will increase by a factor of 300 -- not 300%, but 300 times. By 2015, the storage vendor says, we'll be sitting on a collective 7,910 exabytes. Now, it's true that number likely reflects some aspirational thinking. But it's also true that in 2011, when EMC last issued a digital universe update, the Internet of things wasn't mainstream, and few of us thought much about instrumenting every corner of our worlds and saving every scrap of data for time eternal, the way we are now.

For the sake of argument, let's take EMC at its word. Even if it's off by a factor of 10, the implications are profound on a number of levels -- cost, analysis methods, security, management, how cloud plays in. Companies that are sitting tight waiting for that mythical perfect big data wave should know that many of these areas have matured. Don't hesitate too long: Emerging tools now put analysis within reach, without drowning users in complexity, and costs can be kept under control. There still are risks in investing in emerging big data analytics, but there's also risk in missing business opportunities that might have bubbled up from those oceans of data.

Report Cover
Our report on "6 Big Data Lies" is free with registration. This report includes 36 pages of action-oriented analysis, packed with 25 charts.

What you'll find:
  • Table of the likely annual data volume for a $250 million, 500-person organization
  • Three "little" big data lies IT must address (including security)
Get This And All Our Reports

The thing that scares IT pros most about big data is that it sounds darned expensive. In our InformationWeek 2013 Big Data Survey, when asked about the top barrier to successful use of big data, the No. 1 answer by far was budget constraints, at 38%. Just 13% cite a lack of a business interest, the No. 2 response.

One factor to remember is that the predominant vendor of the previous era usually isn't the leader in the next era. Cost drives that changing of the guard whenever users want to do [insert new IT trend] right this minute but find it's prohibitively expensive with current-generation stuff. Mainframes were too expensive to support most scientific and engineering needs, so we got minis and workstations. Those were too expensive for personal productivity, so enter the PC. PCs weren't portable enough, so $500 tablets appeared and suddenly we're all supporting iPads.

Software seems to move more slowly than hardware in these shifts, sometimes taking decades for a transformation away from a dominant technology. But when it comes to managing and analyzing big data, that kind of important transition is happening right now. Everything is in transition, from servers to endpoints, from dominant operating systems to databases to presentation technology. It's also why now is simultaneously an exciting time for IT pros who like cruising a wave of change and a scary ride for those who prefer to stay safely on shore.

To read the rest of the article,
download the InformationWeek October special issue on big data analytics.

Editor's Choice
Brian T. Horowitz, Contributing Reporter
Samuel Greengard, Contributing Reporter
Nathan Eddy, Freelance Writer
Brandon Taylor, Digital Editorial Program Manager
Jessica Davis, Senior Editor
Cynthia Harvey, Freelance Journalist, InformationWeek
Sara Peters, Editor-in-Chief, InformationWeek / Network Computing