How Real is Real Time?

The demand for real-time analytics is increasing as a result of mobile and the Internet of Things. IT departments have been getting requests for "real-time" analytics for years from various places in the enterprise, and the requests have been triaged based on needs and costs. Because processing and bandwidth costs continue to fall and their capabilities continue to rise, it's now possible to accelerate more types of analytics. However, who actually needs real-time analytics?

Myth or reality? According to a recent press release by OpsClarity, 92% of companies in a recent survey said they are increasing their investment in the real-time analysis of human and machine-generated data and that there's more demand for "instant analysis." My first reaction when reading that was, "These aren't the people I've been talking to," which is absolutely correct. I've been talking to the C suite (including technical titles) and non-vendor consultant department heads. The OpsClaritysurvey primarily involved developers, data architects, and DevOps engineers, 92% of whom said their companies were investing in streaming applications. Although the total number of respondents was not mentioned (which I consider a yellow flag, methodologically speaking), streaming apps will clearly drive more demand for real-time analytics.

The question is, how real is real-time analytics? I surveyed several hundred global executives in 2012 and asked them what they considered real-time. Their responses ranged from less than a second to more than 24 hours. Apparently, not much has changed since then.

In the OpsClarity survey,"real-time" definitions range from less than 30 seconds to less than 24 hours. Near real-time is not defined. In my experience, "near real time" means somewhat slower than whatever someone's real-time definition is. Fuzzy? You bet. So, all this demand for "real-time" analytics means…what? Faster than before. If you're on the razor's edge, you care about microseconds (such as in securities trading).

The ambiguity means, if the IT department is still in charge, it needs to figure out what people mean by "real-time" when they ask for it, rather than responding with a canned "no" response. The same goes for business units and departments that are making their own purchases, because differences in time can make a competitive difference, depending on the use case. Just ask any Wall Street firm or security firm. Microseconds and milliseconds are expensive. Minutes and hours are much more affordable. Obviously, there are trade-offs that need to be considered. In other words, a cost-benefit analysis is wise once one understands the options and actual requirements of the use case.

Why definitions differ There's a lot of hype about real-time analytics which, like any high-profile technology advancement has two effects: one is that more people think they need it whether they actually do or not and the other is people who are not actually mired in developing, implementing, or using technology start throwing around nomenclature without understanding its meaning. The end result in this case, is that real-time means as fast as one needs it which could mean by noon, the end of the day, tomorrow morning, 20 minutes from now, or instantly.