Big Data // Big Data Analytics
News
11/20/2013
04:00 PM
Connect Directly
Google+
RSS
E-Mail

Does Hadoop Have a Speed Problem?

"Fast data" -- information provided in near-real time -- is key to getting the most out of analytics projects, says ParStream CEO Michael Hummel.

ParStream offers a real-time database for big-data analytics.
ParStream offers a real-time database for big-data analytics.

Comment  | 
Print  | 
Comments
Newest First  |  Oldest First  |  Threaded View
MHummel
50%
50%
MHummel,
User Rank: Apprentice
11/22/2013 | 6:19:31 PM
Re: How fast is too fast?
Thomas, I don't think that fast data is the problem here; it's the automated responses that can become problematic. The data speed is simply an enabler.

Wall Street experienced problems because their automated trading systems not only recommend actions but also execute them – all without human interjection.

To obviate any complications, and until these automated systems learn to think about the consequences of their actions, we can rely on the recommendations but should always have decision-makers involved in the execution.
HM
50%
50%
HM,
User Rank: Strategist
11/21/2013 | 3:30:33 PM
Re: Big Data
Jeff, HPCC Systems is an open source data-intensive supercomputing platform to process and solve Big Data analytical problems and is extremely good at what it was designed to do - Massive ingestion of semi structured (or unstructured) data, converting to a normalized form (like RDBMS tables) and performing analytics in an easy to use SQL like language (much more powerful than SQL). In addition, it cleanly integrates with Apache Kafka to provide near real time analytics. More info at http://hpccsystems.com.
virsingh211
50%
50%
virsingh211,
User Rank: Strategist
11/21/2013 | 6:56:55 AM
Re: How fast is too fast?
I agree you Samicksha, but for Hadoop fans one thing to be noted is, Hadoop provides no security model, i.e. it cannot detect a man in the middle attack between nodes.
samicksha
100%
0%
samicksha,
User Rank: Strategist
11/21/2013 | 2:59:57 AM
Re: How fast is too fast?
Impressive part of Hadoop, it is flexible and you can design the cluster size to make task done accordingly.
Thomas Claburn
50%
50%
Thomas Claburn,
User Rank: Author
11/20/2013 | 4:34:46 PM
How fast is too fast?
Given the problems Wall Street has had with automated trades, I wonder whether web sites that rely on rapid-fire analytics data and automated responses will self-optimize too quickly, magnifying problems rather than solving them.
6 Tools to Protect Big Data
6 Tools to Protect Big Data
Most IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest - July10, 2014
When selecting servers to support analytics, consider data center capacity, storage, and computational intensity.
Flash Poll
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
Join InformationWeek’s Lorna Garey and Mike Healey, president of Yeoman Technology Group, an engineering and research firm focused on maximizing technology investments, to discuss the right way to go digital.
Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.