Interesting left-handed compliments here
James doesn't really sound all that enthusiastic about Spark, describing it as unproven and underplaying, in my view, the interest, adoption, and number of proven applications on the platform. Interestingly, IBM sponsored Spark Summit East a couple of weeks ago, but it's probably hoping to sell its own commercial solutions to the attendees of that event. It is officially supporting Spark, but it doesn't have a distribution of or integration with that software as yet.
As I understand it, Spark has more than 500 enterprise adopters, and Spark promoter Databricks has more than 50 beta customers for its Databricks Cloud service based on Spark. Streaming data analysis is just one play for Spark, which makes it a competitor to IBM InfoSphere Streams. How "proven" is Streams, I wonder, and how many customers does it have? Is InfoSphere Streams really getting into the same conversations as Spark and Storm? Big data practitioners seem to have a strong bias toward open-source options, not commercial software. Maybe open source is the real "shiny new thing" that commercial vendors are competing against.