What does "real time" analytics mean to you and your business?
IT departments have been bombarded with requests for real-time analytics capabilities for years, although not everyone who asks for it actually needs it -- or even knows what it means. Is the high cost of moving to real-time analytics justified? Although the prices of memory, storage, and bandwidth all continue to fall, there are technology integration issues, process issues, and cultural issues to be considered.
Adding to the confusion, there is no standardized definition of "real time." Depending on whom you ask, "real-time" can be measured in anything from sub-seconds to a span of more than 24 hours.
Technology innovations such as in-memory computing, Hadoop, and Spark are all designed to address the insatiable need for speed. Yet, a number of persistent issues keep companies from moving as fast as business leaders might desire.
[What's hot in data science? Read 10 Programming Languages and Tools Data Scientists Use Now.]
"IT organizations often do not have the platforms in place to collect, manage, and respond to real-time data. This means any attempt at real-time [action] will be flawed and even dangerous," said Graham Clark, head of digital services for global IT solutions service provider NIIT Technologies, in an interview. "Additionally, creating and using platforms for real-time analytics requires a real-time business mentality, which needs to permeate throughout the organization."
Companies should consider the use-case instead of focusing on real-time analytics as the goal for all projects, said Jeff Veis, VP of big data platform marketing for HPE Software at Hewlett Packard Enterprise, in an interview. "After all, it largely depends on how quickly you need an answer, if the business can wait for that insight -- and at what cost."
Increasingly, businesses are considering the cost of not implementing real-time analytics. These days, response time is considered a competitive weapon, and some observers believe this trend will become even more pronounced with the proliferation of Internet of Things (IoT) devices. However, accelerating business processes introduces risks that must also be weighed against the potential benefits.
"People might go to extremes and demand all available data to be collected and used on a real-time basis, [but] not all data is equally important," said Vivian Zhang, founder and CTO of the NYC Data Science Academy, in an interview. "Some data is critical, some data isn't. When you put too much noise or irrelevant information into the data pipeline, it might not help and it can result in a false impression of what is happening inside the business."
In short, don't underestimate what it takes to do real-time analytics right, because there's a lot that can go wrong along the way. Following are a few of the most common barriers to success. Once you've reviewed these, tell us about your own experiences.
What does "real-time" analytics mean to your organization? How close are you to achieving it? What pitfalls have you encountered? Tell us all about it in the comments section below.