Spark Spreads, Apache Arrow Accepted: Big Data Roundup - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Data Management // Big Data Analytics
News
2/21/2016
10:06 AM
Connect Directly
Twitter
RSS
E-Mail
50%
50%

Spark Spreads, Apache Arrow Accepted: Big Data Roundup

Databricks announced a free community edition of Spark along with free training materials. Apache Arrow became a project within the Apache Software Foundation. And SAP announced support for Spark in its Predictive Analytics platform. We've got that and more in our big data roundup for the week of Feb. 21, 2016.

Where 2016 US Presidential Contenders Stand On Tech Issues
Where 2016 US Presidential Contenders Stand On Tech Issues
(Click image for larger view and slideshow.)

It's been a busy week in big data land. We've got news about a free community edition of Apache Spark plus more news from Spark distributor Databricks, a new Apache Software Foundation project for big data called Arrow, Gartner's Magic Quadrant for Advanced Analytics, and more.

Llet's start with the news from Databricks, the main commercial distributor of Apache Spark. This week at the Spark Summit East in New York, the company rolled out a beta release of Databricks Community Edition, a free version of the cloud-based big data platform. It comes with a set of training resources, including a massive open online course (MOOC), "Introduction to Big Data with Apache Spark."

According to Databricks, the new service provides data scientists and IT pros with the technology they need to get started with Spark, including access to a microcluster and a cluster manager and notebook environment. The free version will be generally available in the second quarter.

[ Want the tech perspective on the Super Bowl? Read NFL CIO: Super Bowl 50 Tech Was A Game Changer. ]

Databricks said it will continue to develop Spark tutorials and training materials to be part of the Community Edition over time.

"As developers at heart, we find value in empowering professionals to tackle big data problems, and as a result, we are committed to the development of the Spark engine and the healthy growth of the community," said Ion Stoica, executive chairman at Databricks, in a prepared statement. "We're happy to contribute back to the community by releasing Community Edition of Databricks for free and we're excited to see how users experiment with the platform."

(Image: PonyWang/iStockphoto)

(Image: PonyWang/iStockphoto)

During Spark Summit East, Databricks also launched Databricks Dashboards as an expansion to its enterprise Spark platform. Databricks said the Dashboards are intended to enable data pros to transform complex results into visual formats that are easy for business users to consume.

Speaking of Spark, SAP announced support for the technology in its Predictive Analytics 2.5 platform, and also announced the acquisition of a mobile visualization company to enhance its advanced analytics portfolio.

Apache Arrow

Apache Arrow has been accepted as a full-fledged project by the Apache Software Foundation. The technology is designed to improve the performance and speed of big data components that work together as part of a larger system.

The project is backed by Tomer Shiran and Jacques Nadeau, the founders of Dremio, who are also the force behind Apache Drill. The technology is designed to enable various projects within the big data Hadoop ecosystem to talk to each other more easily, and to enable multiple development languages to work with the ecosystem.

Magic Quadrant, CDO, And Privacy Shield

Gartner released its 2016 Magic Quadrant report for Advanced Analytics, and we covered the details in a report this week. Plus, we've got an update on the state of the chief data officer, and a look at best approaches for enterprises to take as they await interpretations of the EU-US Privacy Shield.

Cognitive Computing Competition Prize: $5 Million

Finally this week, we've got some news about a new Cognitive Computing Competition. The IBM Watson AI XPRIZE is a $5 million competition challenging teams to develop and demonstrate how humans can collaborate with cognitive technologies to tackle the world's greatest challenges.

Every year leading up to TED2020, teams will go head-to-head at IBM's World of Watson annual conference to compete for interim prizes and the chance to advance to the next year's competition. Three finalist teams will deliver TED Talks in 2020 to provide demonstrations of what they have achieved, according to the competition's website.

A panel of judges will evaluate ideas for technical validity, and the winners will be chosen by TED and XPRISE communities "based on the audacity of their mission and the awe-inspiring nature of the teams' TED Talks in 2020."

IBM said it believes the competition can accelerate the creation of landmark breakthroughs.

What have you done to advance the cause of Women in IT? Submit your entry now for InformationWeek's Women in IT Award. Full details and a submission form can be found here.

Jessica Davis has spent a career covering the intersection of business and technology at titles including IDG's Infoworld, Ziff Davis Enterprise's eWeek and Channel Insider, and Penton Technology's MSPmentor. She's passionate about the practical use of business intelligence, ... View Full Bio

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Commentary
Get Your Enterprise Ready for 5G
Mary E. Shacklett, Mary E. Shacklett,  1/14/2020
Commentary
Modern App Dev: An Enterprise Guide
Cathleen Gagne, Managing Editor, InformationWeek,  1/5/2020
Slideshows
9 Ways to Improve IT and Operational Efficiencies in 2020
Cynthia Harvey, Freelance Journalist, InformationWeek,  1/2/2020
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
The Cloud Gets Ready for the 20's
This IT Trend Report explores how cloud computing is being shaped for the next phase in its maturation. It will help enterprise IT decision makers and business leaders understand some of the key trends reflected emerging cloud concepts and technologies, and in enterprise cloud usage patterns. Get it today!
Slideshows
Flash Poll