Databricks Cloud: Next Step For Spark - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Comments
Databricks Cloud: Next Step For Spark
Oldest First  |  Newest First  |  Threaded View
D. Henschen
50%
50%
D. Henschen,
User Rank: Author
6/30/2014 | 3:27:16 PM
Black Widow of big data technology?
Databricks has partnered with every single Hadoop distributor, all of which have sung Spark's praises. But they also tend to describe Spark as something just for in-memory machine learning. I'm sure they're not thrilled to hear Databricks suggesting that Spark can replace the likes of Hive, Impala, and quite a few other analysis options that run on top of Hadoop. Is it a case of biting the head off your mate after consumating a partnership?
Li Tan
50%
50%
Li Tan,
User Rank: Ninja
7/1/2014 | 9:42:59 AM
Re: Black Widow of big data technology?
At the current stage I will keep my finger crossed and see what will happen next with Spark. From the information available so far, Spark is trying to improve on big data analytics by providing a neat and straightforward data pipeline. But how it performs compared to Hadoop? Is there any live case available so far?
D. Henschen
50%
50%
D. Henschen,
User Rank: Author
7/1/2014 | 11:58:43 AM
Re: Black Widow of big data technology?
Again, it's not a case of Spark vs. Hadoop. Spark runs on Hadoop, but it uses only its most basic storage functionality. My "bite their heads off" comment hints at the value in all the data-analysis stuff on top of Hadoop. Cloudera, for example, has Impala, a partically commercial offering, that Spark could replace. The data-analysis done on top of Hadoop is the valuable high ground -- read, profit-driver for a Hadoop distributor and support provider.


2020 State of DevOps Report
2020 State of DevOps Report
Download this report today to learn more about the key tools and technologies being utilized, and how organizations deal with the cultural and process changes that DevOps brings. The report also examines the barriers organizations face, as well as the rewards from DevOps including faster application delivery, higher quality products, and quicker recovery from errors in production.
Slideshows
10 Trends Accelerating Edge Computing
Cynthia Harvey, Freelance Journalist, InformationWeek,  10/8/2020
Commentary
Is Cloud Migration a Path to Carbon Footprint Reduction?
Joao-Pierre S. Ruth, Senior Writer,  10/5/2020
News
IT Spending, Priorities, Projects: What's Ahead in 2021
Jessica Davis, Senior Editor, Enterprise Apps,  10/2/2020
Register for InformationWeek Newsletters
Video
Current Issue
[Special Report] Edge Computing: An IT Platform for the New Enterprise
Edge computing is poised to make a major splash within the next generation of corporate IT architectures. Here's what you need to know!
White Papers
Slideshows
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Sponsored Video
Flash Poll