Big Data // Big Data Analytics
Commentary
8/20/2014
10:26 AM
Jeff Kaplan
Jeff Kaplan
Commentary
Connect Directly
RSS
E-Mail
50%
50%

When Internet Of Things Meets Big Data

IoT will exponentially increase the volume, variety, and velocity of data, but that's not all you need to worry about.

10 More Big Data Pros To Follow On Twitter
10 More Big Data Pros To Follow On Twitter
(Click image for larger view and slideshow.)

Just as organizations begin to catch their breath and get a grip on current big data challenges, a new wave is coming at us. I'm talking about the rapidly evolving Internet of Things marketplace and its attendant new business opportunities.

As always, the burden will fall on IT to solve the data storage, integration, and analytics dilemmas created by the IoT. You can't use your current strategy because the data to be captured, managed, and exploited will be even more diverse, and the use cases even more varied. The data will come from a variety of sensors attached to a wide array of devices and objects. It will be used for an assortment of reactive, proactive, and, let's say, creative purposes.

IT's job will be to work with business units to fully understand their IoT use cases and match the right technologies to their needs. In particular, IT must identify the best analytics platforms and tools to enable business users to acquire the data they need, analyze its meaning, and act on it quickly. While the range of analytics options available to accommodate big data in general seems broad, the number of systems that can meet the IoT test are more limited.

Two companies attacking the IoT big data challenge illustrate key problems that are not often discussed: query performance and the need for automation to provide an early warning of problems.

[InformationWeek's IoT report focuses on scenarios businesses large and small might use to increase reliability and efficiency, reduce costs, improve the customer ­experience, even build new services.]

Infobright's Knowledge Grid architecture can serve as an IoT database platform for storing and analyzing machine-generated data. The system includes Data Packs, which increase compression ratios from 10:1 to up to 40:1. Compression allows organizations to increase load speeds and query performance times even when data volumes rise dramatically. It also permits them to store the terabytes and petabytes of historical data needed for extended time-duration-oriented analytics, or just because no one wants to take responsibility for deleting data that might one day come in handy.

Infobright's platform is being employed by REZ-1, a software and services firm that helps transportation companies track and manage their assets, inventory, and finances. REZ-1 can now handle 1.5 million rows of data generated daily, store 2.5 billion rows of data, and reduce data query time from hours to less than one minute in most cases. Infobright's platform is also being used in the energy industry to create smart grids that allow energy companies to accurately measure usage patterns so they can more cost-effectively allocate resources and manage their operations.

Another critical use case for IoT is to warn organizations when a product or service is at risk. Early detection is essential to either remedy the issue before it becomes a real problem or quickly do cleanup when failure hits. Prelert is a Boston-area company that uses advanced machine learning analytics to notify users of anomalies in streaming data that indicate operational risks. Anomaly detection is becoming increasing appealing to enterprise and government organizations that are dealing with massive streams of data that need to be analyzed quickly by non-data-scientists with minimal upfront modeling. Prelert's analytic engine is being used to support a wide range of IoT use cases, such as traffic congestion management and wind turbine sensor monitoring.

These are just two examples of a new generation of analytics options available to IT that are specifically designed to respond to the unique challenges and opportunities created by the IoT. Before the tsunami hits, think about what matters to the business beyond just variety.

In its ninth year, Interop New York (Sept. 29 to Oct. 3) is the premier event for the Northeast IT market. Strongly represented vertical industries include financial services, government, and education. Join more than 5,000 attendees to learn about IT leadership, cloud, collaboration, infrastructure, mobility, risk management and security, and SDN, as well as explore 125 exhibitors' offerings. Register with Discount Code MPIWK to save $200 off Total Access & Conference Passes.

Jeff Kaplan is the Managing Director of THINKstrategies (www.thinkstrategies.com), the only strategic consulting firm focused entirely on the business implications of the transition of the technology industry from a product-centric to a services-driven solution model, ... View Full Bio
Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Jeff Jerome
50%
50%
Jeff Jerome,
User Rank: Ninja
8/31/2014 | 5:45:09 PM
Re: Challenge that demands attention

It seems that ask search is so mature now, data is abundant and relevant and with the name internet of things how can we go wrong.  Plus data centers with abundance of cheap storage are everywhere.  It will be a great ride and will help us to solve problems and human needs with that big data and IoT.

pfretty
50%
50%
pfretty,
User Rank: Moderator
8/28/2014 | 11:59:41 AM
Challenge that demands attention
Fortunately at this point both technologies are still in growth/maturity stages. However, as you mention IoT promises to introduce hoards of new data sources with unimaginable volumes attached.  At the same time in a recent IDG SAS survey, most organizations admit they struggle with data collection, cleaning and storage tasks with only ten percent saying they are extremely capable.  Bottom line there needs to be more attention on the details if IT wants to avoid a nightmare a few years down the line. 

 

Peter Fretty
6 Tools to Protect Big Data
6 Tools to Protect Big Data
Most IT teams have their conventional databases covered in terms of security and business continuity. But as we enter the era of big data, Hadoop, and NoSQL, protection schemes need to evolve. In fact, big data could drive the next big security strategy shift.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Tech Digest - September 17, 2014
It doesn't matter whether your e-commerce D-Day is Black Friday, tax day, or some random Thursday when a post goes viral. Your websites need to be ready.
Flash Poll
Video
Slideshows
Twitter Feed
InformationWeek Radio
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.