Apache Spark 2.0 Preview, Google's Amazon Echo Rival: Big Data Roundup - InformationWeek
IoT
IoT
Data Management // Big Data Analytics
News
5/15/2016
11:06 AM
Connect Directly
Twitter
RSS
E-Mail
50%
50%
RELATED EVENTS
Ransomware: Latest Developments & How to Defend Against Them
Nov 01, 2017
Ransomware is one of the fastest growing types of malware, and new breeds that escalate quickly ar ...Read More>>

Apache Spark 2.0 Preview, Google's Amazon Echo Rival: Big Data Roundup

Apache Spark 2.0 preview is released for Databricks customers. Google preps a stationary personal assistant like Amazon Echo. MarkLogic revs up security and encryption. We have all this and more in our Big Data Roundup for the week ending May 15, 2016.

12 Inspiring Women In Data Science, Big Data
12 Inspiring Women In Data Science, Big Data
(Click image for larger view and slideshow.)

Apache Spark 2.0 is coming. Google preps a competitor to Amazon Echo, and gets clever with artificial intelligence (AI) and open source. MarkLogic updates security and adds the Optic API. We've got those all those stories, plus a look at something called the Animosity Index, in this week's Big Data Roundup

Let's start with the next big update to Apache Spark.

Spark is the new sweetheart of the big data world, offering real-time and streaming capabilities to massive amounts of data. This week, a couple of the big distributors of Spark -- Databricks and MapR -- announced plans to release an early version of Apache Spark 2.0.

In a Databricks blog, Reynold Xin said a preview of Apache Spark 2.0 is now available on the Databricks Community Edition.

[Read a classic big data use-case: The Weather Company Brings Together Forecasting and IoT.]

"Since Spark 1.0 came out two years ago, we have heard praises and complaints," Xin wrote. "Spark 2.0 builds on what we have learned in the past two years, doubling down on what users love and improving on what users lament."

(Image: PonyWang/iStockphoto)

(Image: PonyWang/iStockphoto)

Xin noted the official Apache Spark 2.0 release is still a few weeks away, but the technical preview provides early access to those who can't wait. The new version provides easier SQL and streamlined APIs (including unifying data frames and datasets in Scala/Java); performance optimizations that have made Spark up to 10 times faster, depending on the task; and a smarter take on structured streaming through version 2.0's structured streaming APIs.

MapR released Apache Spark 1.6.1 on the MapR Converged Data Platform this week, too. "We have seen a significant customer adoption of Spark for building data pipelines and advanced analytics," said Anoop Dawar, VP of product management for Spark and Hadoop at MapR, in a prepared statement. MapR said the new release offers improved performance gains, persistence of machine learning pipelines, and a new experimental interface called Dataset API.

Google's AI Plans

As developers get ready to head to Google I/O in the coming week, word on the street is that the technology giant is preparing to announce a competitor to Amazon Echo.

Amazon's home device acts as a stationary personal assistant, equipped with a speaker and microphone, which can perform voice interaction, music playback, weather reports, and more. Google is expected to unveil a rival to this device in the days ahead that will rely on spoken commands and queries to perform searches and seek assistance. No word on the official name of this device, but the internal code word is reportedly "Chip." Whatever the name, we hope Google comes up with something slightly catchier than Google Now.

This past week, Google was focused more on the development backend of its own artificial intelligence efforts. The company released SyntaxNet, its natural language parsing framework, to open source. The technology is a neural network framework implemented in Google's Tensor Flow. It's designed to perform the tasks that human linguists do, such as tagging parts of speech, identifying syntactic dependencies, and sentence compression.

MarkLogic 9

This NoSQL database company has been around for several years. It has been working closely with the healthcare vertical the past few years, and is part of the turnaround success story behind the Healthcare.gov site that implemented the Affordable Care Act.

Version 9 of the software increases security in several ways, including adding encryption to the core of the database. It also offers the new Optic API, which allows analysts to view data based on the problem at hand.

The Animosity Index

Finally this week, as we head into the home stretch of the US Presidential Primary races, it's not surprising that people are a little bit rattled. We may all be feeling a little bit angrier than usual.

To keep track of this, Logz.io has created the Animosity Index, a piece of its overall 2016 US Election Real-Time Dashboard. The Animosity Index tracks "the number of times the words "f*** you" are tweeted to each candidate. Other indexes featured on the dashboard are the Liar Index, the top Execution Topics Index, the Honesty Index, and the Trump Geo Index. That last measure provides the location of people who mention Trump's name or his Twitter handle in their tweets.

Jessica Davis has spent a career covering the intersection of business and technology at titles including IDG's Infoworld, Ziff Davis Enterprise's eWeek and Channel Insider, and Penton Technology's MSPmentor. She's passionate about the practical use of business intelligence, ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
StevenU837
50%
50%
StevenU837,
User Rank: Apprentice
5/18/2016 | 1:02:47 AM
Google Home ...
I love my Amazon Echo.  Actualy I have two and a third (the dot) on order.  We have a bed & breakfasts and our guests are amazed ... and want to go home and order one for themselves.  The integration into the Amazon echo system is good, includign reading Kinidle and Audible books. When I ask guests what artist they like and then, for example, tell Alexa to play "Neil Young on Prime" the love it.  And I point out that I don't even own the music -- its included in my Primer Membership.    I don't yet use it for home automation, but it replaced my alarm clock and radio on my night stand.  There are a few things I would like my Echo to do that it does not currently do, but new things come online monthly.

While I love Alexa, I am also interested to see what Google comes up with.  I also love my Samsung Galaxy Note 4 and will be moving to the Note 6 once it becomes available.  I am just not clear what Google will do to differentiate itself from Alexa.  I am not clear that Google has the conent access tht Amazon has.

Actually if Apple produced Siri in a box I think Apple could have a winner for Apple users who just prefer Apple.  
How Enterprises Are Attacking the IT Security Enterprise
How Enterprises Are Attacking the IT Security Enterprise
To learn more about what organizations are doing to tackle attacks and threats we surveyed a group of 300 IT and infosec professionals to find out what their biggest IT security challenges are and what they're doing to defend against today's threats. Download the report to see what they're saying.
Register for InformationWeek Newsletters
White Papers
Current Issue
2017 State of IT Report
In today's technology-driven world, "innovation" has become a basic expectation. IT leaders are tasked with making technical magic, improving customer experience, and boosting the bottom line -- yet often without any increase to the IT budget. How are organizations striking the balance between new initiatives and cost control? Download our report to learn about the biggest challenges and how savvy IT executives are overcoming them.
Video
Slideshows
Twitter Feed
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.
Flash Poll