Big Data // Software Platforms
News
4/7/2014
03:00 PM
Connect Directly
LinkedIn
Twitter
Google+
RSS
E-Mail

Teradata QueryGrid: Beyond Enterprise Data Warehouse

Teradata adds unified fabric access to myriad databases and Hadoop, triggering multiple analysis engines with a single query. Say goodbye to yesterday's enterprise data warehouse ideas.

Comment  | 
Print  | 
Comments
Newest First  |  Oldest First  |  Threaded View
masifabbasi
100%
0%
masifabbasi,
User Rank: Apprentice
5/24/2014 | 4:13:11 PM
Re: Propagating a myth about Hadoop
Very nice article... Thanks for expanding on QueryGrid. Companies like Cisco's composite have built some top-class optimizers that do the similar stuff, but have the edge of 8 years of RnD on Federated queries. Top class rule based optimizers, Intelligent data ships, and an overlay of cost based optimizer makes it really useful for today's myriad of data sets.  Just wanted to respond to some of your comments above:

"DBMS vendors like to say that Hadoop is all about "unstructured" or "semi-structured" information."


I don't think that is what they are saying. Companies like Teradata (IMHO) are saying that Hadoop acting like a data lake is a good place to store structure, unstructured and semi-structured data, which is cold, and near-real-time access isn't a requirement. Hadoop vendors like Hortonworks also agree with the statement.

"Check out my recent profile of Merck & Co., which combined 16 different types of very structured data to figure out why some batches of a vaccine had high yield rates and other batches had low yield rates. The advantage was being able to "dump everything in a lake" without time-consuming data modeling and ETL work."

That is exactly what these companies are proposing. Modelling and ETL aren't required for all use cases in the data management world. We have done POCs where we had done worked with 10-100's of Terabytes of data using a combination of Hadoop & Aster, and acheived results within a couple of weeks.  Essentially these companies are saying that you need a cheap platform to store the data (like Hadoop), but when you want to do number crunching and complex analytics, you need powerful and intelligent SQL, MapReduce and Graph Engines (unless you are Facebook,Google, LinkedIn having access to top-tiers of talent), and use your skillset to acheive the result set.



Having said all that, majority of the problems out there could be solved using a multitude of technologies. The decision to use a particular technology depends/should depend on the following:

1. Urgency of problem (SLA)

2. Your skillset/Strength

3. Skillset available in the market

4. ROI

Thanks for sharing the article BTW :)

 

 
D. Henschen
50%
50%
D. Henschen,
User Rank: Author
4/7/2014 | 5:50:10 PM
Propagating a myth about Hadoop
RDBMS vendors like to say that Hadoop is all about "unstructured" or "semi-structured" information. And their favorite example is Twitter and Facebook comments. That's not the full truth and it's a bit of a left-handed compliment meant to diminish the value of Hadoop. The fact is, Hadoop can handle very structured information that happens to be varied, voluminuous, inconsistent (a.k.a. sparse) or all of the above.

Check out my recent profile of Merck & Co., which combined 16 different types of very structured data to figure out why some batches of a vaccine had high yield rates and other batches had low yield rates. The advantage was being able to "dump everything in a lake" without time-consuming data modeling and ETL work. Within three months Merck was able to cluster and visualize batch yield rates and spot "smoking guns" within 10 years worth of product and manufacturing plant data. The data wasn't actually all that big -- only 1.5 terabytes -- but the ability to bring together a variety of data quickly made all the difference.
In A Fever For Big Data
In A Fever For Big Data
Healthcare orgs are relentlessly accumulating data, and a growing array of tools are becoming available to manage it.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest - August 27, 2014
Who wins in cloud price wars? Short answer: not IT. Enterprises don't want bare-bones IaaS. Providers must focus on support, not undercutting rivals.
Flash Poll
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
Howard Marks talks about steps to take in choosing the right cloud storage solutions for your IT problems
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.