Turbo Charging Analytics for Faster Decisions - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Data Management // Big Data Analytics
Commentary
12/10/2015
02:00 PM
Connect Directly
Twitter
RSS
50%
50%

Turbo Charging Analytics for Faster Decisions

With the focus increasingly on real-time analytics, there are some new infrastructure technologies that are doing their part.

Phu Hoang, who originally built Hadoop-based systems at Yahoo before co-founding the data batch and streaming company Data Torrent, understands the limitations of running a batch-only platform like Hadoop and the need that many enterprises have for real time analytics, or at least analytics that give them faster mean times to decision.

“People who are seeing demonstrations of it, proofs of concepts, are starting to really face that incredible opportunity of ‘Oh my goodness, the data I’ve been computing, which I was getting once day or every eight hours, I literally can have in minutes or seconds,'” says Hoang.

Real-time analytics make a difference in logistics. Credit: Flickr
Real-time analytics make a difference in logistics.

Credit: Flickr

This real time and near real time analytics revolution is already in motion with compelling use cases.

  • Financial services companies use real time fraud detection systems to intercept fraud by detecting it before it happens. They do this by analyzing in real time the usage patterns of their customers, and by flagging any usage occurrence that deviates from these patterns and then instantly locking down a card at the register so the transaction can’t go through. The stakes are high. In the U.S. alone, 31.8 million persons had their credit cards breached in 2014.
  • Logistics companies track goods in transport in real time. They can also observe failures of temperature and humidity controls in closed shipment containers, as well as tampering or breakage of container seals. These real time analytics that are driven by sensor-generated machine data flowing over the Internet are filtered and analyzed in real time, enabling companies to take immediate action on situations that threaten cargo half a world away. In one case, a major oil and gas exploration company operating in Africa reduced cargo thefts on transport trucks from 50 percent to four percent with the attachment of sensors to payloads. Needless to say, operating expenses and time to market both benefitted.
  • Retailers use clickstream analytics to launch instantaneous promotions and sales offers. They capture the customer’s interest and wallet at the point of sale, and sell even more. Retailers can also track inventory and instantly replenish supplies in areas that demonstrate high demand.

However, while all of these are convincing business use cases, the use of real time or near real time analytics is only as good as the underlying technologies that can deliver them. This is why it is encouraging to see some of the recent breakthroughs in hardware and software that will facilitate it.

Here are some of these technological developments.

New semiconductor technology speeds big data processing and can bypass the ETL stage of preparing big data. Field Programmable Gate Arrays (FPGAs) can replace many of the functions of standard x86 style processor chips and also eliminate the need to string together multiple x86 processors so that enough power can be clustered together to propel a real time or near real time data analytics effort. When FPGA technology is combined with data that is striped across solid state disk (SSD) to increase processing speeds, raw data flowing into the enterprise from the network’s edge can move rapidly into the hands of business users, and be resident on a single box. The technology still uses an x86 frontend processor, but this frontend processor works in concert with an FPGA fabric that takes on the lion’s share of the processing work, yet is abstracted from the end user, who only has to worry about producing the result of an analytics query that will likely process in half the time that it took him before.

The use of semantic layers on top of Hadoop to make user access easier, faster and more secure. One such method is the online analytical processing (OLAP) cube, which is a logical interface to raw Hadoop that enables business analytics users to easily produce their own queries without having to worry about the underlying complexities of Hadoop. By using a logical semantics layer on top of raw Hadoop, a business user without an IT background can obtain answers to many queries on his own in five seconds or less.

The use of new middleware messaging protocols like data distribution service (DDS). DDS can both publish and accept data from multiple sources in real time and then transport data in real time "moving databases" that enable companies to immediately act upon it.

For any new technology, there are always champions and detractors, with the much of the enterprise community sitting on the sidelines while industry consortia and vendors decide which methodologies will prevail. But regardless of which set of technologies prevail, mean times to decision for enterprises using analytics will get shorter, and it isn’t too soon to start visualizing business cases for real time information that can transform your company.

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
Slideshows
Reflections on Tech in 2019
James M. Connolly, Editorial Director, InformationWeek and Network Computing,  12/9/2019
Slideshows
What Digital Transformation Is (And Isn't)
Cynthia Harvey, Freelance Journalist, InformationWeek,  12/4/2019
Commentary
Watch Out for New Barriers to Faster Software Development
Lisa Morgan, Freelance Writer,  12/3/2019
White Papers
Register for InformationWeek Newsletters
Video
Current Issue
The Cloud Gets Ready for the 20's
This IT Trend Report explores how cloud computing is being shaped for the next phase in its maturation. It will help enterprise IT decision makers and business leaders understand some of the key trends reflected emerging cloud concepts and technologies, and in enterprise cloud usage patterns. Get it today!
Slideshows
Flash Poll