Spark Spreads, Apache Arrow Accepted: Big Data Roundup
Databricks announced a free community edition of Spark along with free training materials. Apache Arrow became a project within the Apache Software Foundation. And SAP announced support for Spark in its Predictive Analytics platform. We've got that and more in our big data roundup for the week of Feb. 21, 2016.
Where 2016 US Presidential Contenders Stand On Tech Issues
(Click image for larger view and slideshow.)
It's been a busy week in big data land. We've got news about a free community edition of Apache Spark plus more news from Spark distributor Databricks, a new Apache Software Foundation project for big data called Arrow, Gartner's Magic Quadrant for Advanced Analytics, and more.
Llet's start with the news from Databricks, the main commercial distributor of Apache Spark. This week at the Spark Summit East in New York, the company rolled out a beta release of Databricks Community Edition, a free version of the cloud-based big data platform. It comes with a set of training resources, including a massive open online course (MOOC), "Introduction to Big Data with Apache Spark."
According to Databricks, the new service provides data scientists and IT pros with the technology they need to get started with Spark, including access to a microcluster and a cluster manager and notebook environment. The free version will be generally available in the second quarter.
Databricks said it will continue to develop Spark tutorials and training materials to be part of the Community Edition over time.
"As developers at heart, we find value in empowering professionals to tackle big data problems, and as a result, we are committed to the development of the Spark engine and the healthy growth of the community," said Ion Stoica, executive chairman at Databricks, in a prepared statement. "We're happy to contribute back to the community by releasing Community Edition of Databricks for free and we're excited to see how users experiment with the platform."
During Spark Summit East, Databricks also launched Databricks Dashboards as an expansion to its enterprise Spark platform. Databricks said the Dashboards are intended to enable data pros to transform complex results into visual formats that are easy for business users to consume.
The project is backed by Tomer Shiran and Jacques Nadeau, the founders of Dremio, who are also the force behind Apache Drill. The technology is designed to enable various projects within the big data Hadoop ecosystem to talk to each other more easily, and to enable multiple development languages to work with the ecosystem.
Finally this week, we've got some news about a new Cognitive Computing Competition. The IBM Watson AI XPRIZE is a $5 million competition challenging teams to develop and demonstrate how humans can collaborate with cognitive technologies to tackle the world's greatest challenges.
Every year leading up to TED2020, teams will go head-to-head at IBM's World of Watson annual conference to compete for interim prizes and the chance to advance to the next year's competition. Three finalist teams will deliver TED Talks in 2020 to provide demonstrations of what they have achieved, according to the competition's website.
A panel of judges will evaluate ideas for technical validity, and the winners will be chosen by TED and XPRISE communities "based on the audacity of their mission and the awe-inspiring nature of the teams' TED Talks in 2020."
IBM said it believes the competition can accelerate the creation of landmark breakthroughs.
What have you done to advance the cause of Women in IT? Submit your entry now for InformationWeek's Women in IT Award. Full details and a submission form can be found here.
Jessica Davis has spent a career covering the intersection of business and technology at titles including IDG's Infoworld, Ziff Davis Enterprise's eWeek and Channel Insider, and Penton Technology's MSPmentor. She's passionate about the practical use of business intelligence, ... View Full Bio
6 Tools to Protect Big DataMost IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.
Big Data Brings Big Security ProblemsWhy should big data be more difficult to secure? In a word, variety. But the business won’t wait to use it to predict customer behavior, find correlations across disparate data sources, predict fraud or financial risk, and more.
Top IT Trends to Watch in Financial ServicesIT pros at banks, investment houses, insurance companies, and other financial services organizations are focused on a range of issues, from peer-to-peer lending to cybersecurity to performance, agility, and compliance. It all matters.
Join us for a roundup of the top stories on InformationWeek.com for the week of September 18, 2016. We'll be talking with the InformationWeek.com editors and correspondents who brought you the top stories of the week to get the "story behind the story."