IoT
IoT
Data Management // IoT
News
1/25/2016
08:05 AM
100%
0%

Edge Analytics An Antidote To IoT Data Deluge

As more organizations embrace the IoT, some may find themselves overwhelmed by the volume of data generated. Edge analytics can reduce the need to store and process it all at a central location. Here are the details.

6 Machine Learning, AI, Analytics Trends To Watch
6 Machine Learning, AI, Analytics Trends To Watch
(Click image for larger view and slideshow.)

Gartner estimates that the Internet of Things (IoT) will include 26 billion devices by 2020. Organizations in virtually every industry are using IoT devices to drive higher levels of efficiency, reduce costs, generate new revenue, and understand customers at more granular levels. However, not all of these organizations are prepared to deal with the deluge of data these IoT devices will bring.

"IoT deployments will generate large quantities of data that will need to be processed and analyzed in real time," said Gartner research director Fabrizio Biscotti in a statement. "Processing large quantities of IoT data in real time will increase as a proportion of workloads of data centers, leaving providers facing new security, capacity, and analytics challenges."

One way of addressing these challenges is to put automated, intelligent analytics at the edge -- near where the data is generated -- to reduce the amount of data and networking communications overhead.

"We definitely don't want to save all data. Trying to decide what to save and what to use is as important as having the facility to capture it," said Nik Rouda, senior analyst at ESG Research.

Rather than sending every bit of data to a centralized location where it can be analyzed, edge analytics places another layer of intelligence and automation where the data resides.

"Having access to all of your data is important, but with access comes responsibility and a need for a strategy about which data needs to be collected at the atomic level, which data needs to be rolled up and aggregated, and which data needs to be used to run your business," said Shawn Rogers, chief research officer at Dell Statistica.

(Image: Geralt via Pixabay)

(Image: Geralt via Pixabay)

Thinking Differently About Data

Yet, a lot of organizations are saving every bit of data, just in case it might become valuable in the future. From a traditional data center point of view, that's probably not practical over the long term, especially for companies extending out to the IoT.

Some organizations are now creating data lakes to deal with the growing volume of data, whether or not the IoT is part of their strategy. But as always, just because the data can be saved, doesn't necessarily mean it should be saved, unless there's a compelling reason to do so, such as regulatory compliance.

"We're going to see a lot of data collected before people know what to do with it," said Moshe Kranc, CTO of Ness Software Engineering Services. "As the mesh develops and we see all kinds of data being collected and interconnected, there's going to be a need to process it, and it can't possibly be processed in a central location. It has to be processed at the edge."

That's not to say that none of the IoT data will make its way back to a central location, but organizations will have to decide what happens at the edge, at the core, and perhaps in-between. For example, rather than send all sensor data to a central location, an edge device or software solution may send a summary of the data or trigger an automatic alert based a threshold-level status change.

Dell Statistica already has an edge analytics gateway that is capable of communicating with a wide array of sensors. Using its platform users can write a rule-based model that determines how the gateway handles data.

"Our traditional practices have been to collect, harvest, aggregate, and store data, but we're not allowing data to live where it best lives. [Now] we're being more strategic in our approach to the data to decide what data needs to move back to the core and what data can be leveraged for value," said Dell Statistica's Rogers.

Some of the Challenges

Like anything else, the benefits of edge analytics have to be weighed against its costs and risks, which will vary from use-case to use-case.

"Although RFID chips are cheap enough to track boxes in a warehouse, when you're talking millions of boxes, that adds up. And if you think about sensing equipment or a seismograph to study motion in the ground, that can get really expensive if you want to do it over a wide area," said ESG Research's Rouda.

[For more insight on what's coming down the analytics pipeline, see Big Data Goes Mainstream: What Now?]

Security is another issue, because adding another technology layer introduces another potential point of vulnerability. In addition, the data being collected may raise privacy concerns, by itself or when it's combined with other kinds of data. In other words, data governance may need some adjustment.

To be of value, edge analytics should be part of a holistic data strategy. Otherwise, companies run the risk of creating an information silo, which would be a huge step backward from both business and IT perspectives.

"Whenever you acknowledge that data should live where it needs to, you run the risk of creating a siloed environment if you don't have a strategy that includes smart data integration and management, an infrastructure that can support it, and end-to-end security that covers your entire ecosystem," said Rogers.

Bottom Line

The perpetual questions of what data can be collected, what data should be collected, and how long the data should be retained still apply. The difference is the physical point at which the data should be analyzed and acted upon, which depends on the use-case and on what an organization is trying to achieve.

"I think a lot of data can be understood locally. Edge analytics means you don't need to bring everything to the data center and save it forever. Maybe we can interpret it, respond to it, record it, or summarize it, and then share it with other locations," said Rouda.

While we'll likely hear more about edge analytics in the near future, not everyone will agree that it's necessary. As processing power and storage costs continue to fall, and network bandwidth continues to increase, organizations may decide that they can keep every bit of their data regardless. Whether that actually makes sense is a question that organizations can only answer for themselves.

Lisa Morgan is a freelance writer who covers big data and BI for InformationWeek. She has contributed articles, reports, and other types of content to various publications and sites ranging from SD Times to the Economist Intelligent Unit. Frequent areas of coverage include ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
tzubair
50%
50%
tzubair,
User Rank: Ninja
1/27/2016 | 2:42:34 PM
Re: Edge Analytics has massive Potential Going ahead.
"Undoubtedly,if you did better and more localized analysis with all the IoT Data it would save not just a lot of Bandwidth but Processing power at the Central Locations as well."

@Ashu001: It may increase robustness and speed of response, but I doubt if it is going to result in saving costs. If you have analytics systems at the front end, they require huge processing capabilities along with storage. That comes at a fixed cost and must be present for all "edges" so to say. Having a central system with the analytics software is a far cheaper solution. The infrastucture cost is mostly fixed or semi-variable.
tzubair
50%
50%
tzubair,
User Rank: Ninja
1/27/2016 | 2:40:00 PM
Edge analytics
 

"That's not to say that none of the IoT data will make its way back to a central location, but organizations will have to decide what happens at the edge, at the core, and perhaps in-between"

I think analytics is normally performed on data that's available in huge numbers so that a certain pattern can be traced. This is best suited to be done at the back-end after data for a certain time frame has been gathered. At the front-end or the "edge", I wonder what kind of analytics can be performed and whether there's sufficient stroage and processing capacity to perform the necessary analysis.
Ashu001
50%
50%
Ashu001,
User Rank: Ninja
1/25/2016 | 4:07:24 PM
Edge Analytics has massive Potential Going ahead.
Lisa,

I really liked what you had to say here-


The perpetual questions of what data can be collected, what data should be collected, and how long the data should be retained still apply. The difference is the physical point at which the data should be analyzed and acted upon, which depends on the use-case and on what an organization is trying to achieve.

Undoubtedly,if you did better and more localized analysis with all the IoT Data it would save not just a lot of Bandwidth but Processing power at the Central Locations as well.


 
Register for InformationWeek Newsletters
White Papers
Current Issue
Top IT Trends to Watch in Financial Services
IT pros at banks, investment houses, insurance companies, and other financial services organizations are focused on a range of issues, from peer-to-peer lending to cybersecurity to performance, agility, and compliance. It all matters.
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
Join us for a roundup of the top stories on InformationWeek.com for the week of August 21, 2016. We'll be talking with the InformationWeek.com editors and correspondents who brought you the top stories of the week to get the "story behind the story."
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.