IoT
IoT
Data Management // Software Platforms
News
2/3/2016
07:06 AM
Jessica Davis
Jessica Davis
Slideshows
Connect Directly
Twitter
RSS
E-Mail
50%
50%
RELATED EVENTS
Core System Testing: How to Achieve Success
Oct 06, 2016
Property and Casualty Insurers have been investing in modernizing their core systems to provide fl ...Read More>>

Hadoop At 10: Milestones And Momentum

Hadoop, an open source framework for wrangling unstructured data and analytics, celebrated its 10th birthday in January. Here's a look at the milestones, players, and events that marked the growth of this groundbreaking technology.
Previous
1 of 11
Next

(Image: Cloudera)

(Image: Cloudera)

The year was 2006. Facebook was a two-year-old startup company, run by a 21-year-old in a hoodie. Some entrepreneurs had joined together to launch a new social media service called Twitter, and in December the world was still six months away from seeing the introduction of the iPhone. It was a different time.

Consumers weren't carrying around sensor-laden, camera-equipped data collection devices everywhere they went and posting every thought, emotion, and meal to social media. When companies thought about data, they thought about structured data in ERP and CRM systems, and how they could create better business intelligence reports for executives.

It was in this environment that a new technology called Hadoop was born. It started as a framework to support a search engine project called Nutch. Nutch's creators needed a way to store and process the massive amount of data collected for their search engine to use, so they created a new software framework based on inspiration gained from a couple of papers published by growing Silicon Valley upstart Google.

[Check out our interview with Hadoop creator Doug Cutting as he talks about what this birthday means for big data and the software framework. Read Hadoop At 10: Doug Cutting On Making Big Data Work.]

These two developers, Doug Cutting and Mike Cafarella, eventually joined a different company called Yahoo, which then was struggling to retain its lead in the metric of site visits against this upstart Google. At Yahoo, their work on the distributed file system and framework for parallel processing was named Hadoop, after a toy stuffed elephant that Cutting's son had. Yahoo eventually sent Hadoop to open source organization the Apache Foundation. And work continued on making this not-quite-ready-for-prime-time distributed storage and processing system more scalable.

Today, Hadoop has entered a new stage. Load improvements, as well as add-on projects, have turned the software framework into a powerful tool used in a number of big companies, including Facebook, Twitter, eBay, and Salesforce. Hadoop indeed seems like it's getting ready for prime time.

In the Forrester Wave: Big Data Hadoop Distributions, Q1 2016 report, the analyst firm said: "Enterprise Hadoop is a market that is not even 10 years old, but Forrester estimates that 100% of all large enterprises will adopt [Hadoop and related technologies such as Spark] for big data analytics within the next two years."

To celebrate Hadoop's 10-year anniversary, come with us as we look back at some of the milestones, key players, and important developments in Hadoop's history.

Are you a Hadoop user? Is it something you're considering for your enterprise? Is there an important milestone we missed? Tell us all about it in the comments section below.

Rising stars wanted. Are you an IT professional under age 30 who's making a major contribution to the field? Do you know someone who fits that description? Submit your entry now for InformationWeek's Pearl Award. Full details and a submission form can be found here.

Jessica Davis has spent a career covering the intersection of business and technology at titles including IDG's Infoworld, Ziff Davis Enterprise's eWeek and Channel Insider, and Penton Technology's MSPmentor. She's passionate about the practical use of business intelligence, ... View Full Bio

Previous
1 of 11
Next
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Charlie Babcock
50%
50%
Charlie Babcock,
User Rank: Author
2/4/2016 | 7:49:31 PM
Production version is relatively new
Interesting points on Hadoop. I think of Hadoop as one of the earliest and most straight forward examples of how a cloud software system is supposed to work. Hard to believe the 1.0 version was as recent at 2012.
batye
50%
50%
batye,
User Rank: Ninja
2/3/2016 | 9:46:56 AM
Re: 100%
@Ariella I trust you are right, changes in the way, we just not seeing them yet but Hadoop is here to stay and grow... - how I see it...
Ariella
100%
0%
Ariella,
User Rank: Author
2/3/2016 | 8:55:55 AM
100%
I'm used to seeing fairly bullish prediction, but usually there is at least some qualifier involved. I'm surprised they go all out and say 100% here: "Enterprise Hadoop is a market that is not even 10 years old, but Forrester estimates that 100% of all large enterprises will adopt [Hadoop and related technologies such as Spark] for big data analytics within the next two years."
In A Fever For Big Data
In A Fever For Big Data
Healthcare orgs are relentlessly accumulating data, and a growing array of tools are becoming available to manage it.
Register for InformationWeek Newsletters
White Papers
Current Issue
Top IT Trends to Watch in Financial Services
IT pros at banks, investment houses, insurance companies, and other financial services organizations are focused on a range of issues, from peer-to-peer lending to cybersecurity to performance, agility, and compliance. It all matters.
Video
Slideshows
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.