Big Data // Big Data Analytics
04:00 PM
Connect Directly

Does Hadoop Have a Speed Problem?

"Fast data" -- information provided in near-real time -- is key to getting the most out of analytics projects, says ParStream CEO Michael Hummel.

ParStream offers a real-time database for big-data analytics.
ParStream offers a real-time database for big-data analytics.

Comment  | 
Print  | 
Newest First  |  Oldest First  |  Threaded View
User Rank: Apprentice
11/22/2013 | 6:19:31 PM
Re: How fast is too fast?
Thomas, I don't think that fast data is the problem here; it's the automated responses that can become problematic. The data speed is simply an enabler.

Wall Street experienced problems because their automated trading systems not only recommend actions but also execute them – all without human interjection.

To obviate any complications, and until these automated systems learn to think about the consequences of their actions, we can rely on the recommendations but should always have decision-makers involved in the execution.
User Rank: Strategist
11/21/2013 | 3:30:33 PM
Re: Big Data
Jeff, HPCC Systems is an open source data-intensive supercomputing platform to process and solve Big Data analytical problems and is extremely good at what it was designed to do - Massive ingestion of semi structured (or unstructured) data, converting to a normalized form (like RDBMS tables) and performing analytics in an easy to use SQL like language (much more powerful than SQL). In addition, it cleanly integrates with Apache Kafka to provide near real time analytics. More info at
User Rank: Strategist
11/21/2013 | 6:56:55 AM
Re: How fast is too fast?
I agree you Samicksha, but for Hadoop fans one thing to be noted is, Hadoop provides no security model, i.e. it cannot detect a man in the middle attack between nodes.
User Rank: Strategist
11/21/2013 | 2:59:57 AM
Re: How fast is too fast?
Impressive part of Hadoop, it is flexible and you can design the cluster size to make task done accordingly.
Thomas Claburn
Thomas Claburn,
User Rank: Author
11/20/2013 | 4:34:46 PM
How fast is too fast?
Given the problems Wall Street has had with automated trades, I wonder whether web sites that rely on rapid-fire analytics data and automated responses will self-optimize too quickly, magnifying problems rather than solving them.
6 Tools to Protect Big Data
6 Tools to Protect Big Data
Most IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Elite 100 Digital Issue, April 2015
The 27th annual ranking of the leading US users of business technology
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
Join us for a roundup of the top stories on for the week of April 19, 2015.
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.