More than a buzzword
Big data, however you define it, has been praised and vilified. It's many things to many people: a boon to scientists and retailers, but also an enabling technology for a host of privacy and security threats.
Whether savior or scam -- or maybe even a mixture of the two -- big data remains a popular topic among pundits, prognosticators, marketers, and security buffs. Its unofficial definition is evolving as well. So what is it? Wikipedia's description is a good start: "any collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications."
But the challenges of managing massive volumes of varied data sets arriving at high velocities -- the classic 3V's definition -- are changing as the number of data-sharing devices grows exponentially. This hardware, collectively known as the Internet of Things (IoT), includes machine sensors and consumer-oriented devices such as connected thermostats, light bulbs, refrigerators, and wearable health monitors. IDC predicts the IoT market will soar in the coming years -- from 9.1 billion installed units at the end of 2013 to 28.1 billion by 2020.
Organizations see a potential boon in actionable insights derived from big data, not only to sell more widgets and services, but also to better manage healthcare, stop the flow of counterfeit drugs, track terrorists, and maybe even track your phone calls. Hence it's a given that big data isn't inherently good or evil. It's how you use it that counts.
The irony of big data is that despite its potential to enhance the human experience, it's often difficult to collect, filter, analyze, and interpret to gain those cherished insights. This slideshow examines the challenges and capabilities of big data. The facts and figures may surprise you. What to expect? Well, the future appears bright for Hadoop, the leading big data platform. And data scientists and related big data gurus should be gainfully (and lucratively) employed for years to come.
Industry insiders have predicted the buzz term "big data" will fade away. "It is all just data, after all. Big data and all the predictions for this space will collapse into 'data management' by the analysts and all those following, including a lot of the 'big' vendors," wrote Hortonworks president Herb Cunitz in a December 2012 blog.
Cunitz may have prematurely predicted the demise of "big data," but he's spot on: It's all just data. Only the tools needed to manage it will change. Now dig into our slideshow and get a look at some revealing statistics and research.
.svg)