When Internet Of Things Meets Big Data

10 More Big Data Pros To Follow On Twitter

Just as organizations begin to catch their breath and get a grip on current big data challenges, a new wave is coming at us. I'm talking about the rapidly evolving Internet of Things marketplace and its attendant new business opportunities.

As always, the burden will fall on IT to solve the data storage, integration, and analytics dilemmas created by the IoT. You can't use your current strategy because the data to be captured, managed, and exploited will be even more diverse, and the use cases even more varied. The data will come from a variety of sensors attached to a wide array of devices and objects. It will be used for an assortment of reactive, proactive, and, let's say, creative purposes.

IT's job will be to work with business units to fully understand their IoT use cases and match the right technologies to their needs. In particular, IT must identify the best analytics platforms and tools to enable business users to acquire the data they need, analyze its meaning, and act on it quickly. While the range of analytics options available to accommodate big data in general seems broad, the number of systems that can meet the IoT test are more limited.

Two companies attacking the IoT big data challenge illustrate key problems that are not often discussed: query performance and the need for automation to provide an early warning of problems.

[InformationWeek's IoT report focuses on scenarios businesses large and small might use to increase reliability and efficiency, reduce costs, improve the customer ­experience, even build new services.]

Infobright's Knowledge Grid architecture can serve as an IoT database platform for storing and analyzing machine-generated data. The system includes Data Packs, which increase compression ratios from 10:1 to up to 40:1. Compression allows organizations to increase load speeds and query performance times even when data volumes rise dramatically. It also permits them to store the terabytes and petabytes of historical data needed for extended time-duration-oriented analytics, or just because no one wants to take responsibility for deleting data that might one day come in handy.

Infobright's platform is being employed by REZ-1, a software and services firm that helps transportation companies track and manage their assets, inventory, and finances. REZ-1 can now handle 1.5 million rows of data generated daily, store 2.5 billion rows of data, and reduce data query time from hours to less than one minute in most cases. Infobright's platform is also being used in the energy industry to create smart grids that allow energy companies to accurately measure usage patterns so they can more cost-effectively allocate resources and manage their operations.

Another critical use case for IoT is to warn organizations when a product or service is at risk. Early detection is essential to either remedy the issue before it becomes a real problem or quickly do cleanup when failure hits. Prelert is a Boston-area company that uses advanced machine learning analytics to notify users of anomalies in streaming data that indicate operational risks. Anomaly detection is becoming increasing appealing to enterprise and government organizations that are dealing with massive streams of data that need to be analyzed quickly by non-data-scientists with minimal upfront modeling. Prelert's analytic engine is being used to support a wide range of IoT use cases, such as traffic congestion management and wind turbine sensor monitoring.

These are just two examples of a new generation of analytics options available to IT that are specifically designed to respond to the unique challenges and opportunities created by the IoT. Before the tsunami hits, think about what matters to the business beyond just variety.

In its ninth year, Interop New York (Sept. 29 to Oct. 3) is the premier event for the Northeast IT market. Strongly represented vertical industries include financial services, government, and education. Join more than 5,000 attendees to learn about IT leadership, cloud, collaboration, infrastructure, mobility, risk management and security, and SDN, as well as explore 125 exhibitors' offerings. Register with Discount Code MPIWK to save $200 off Total Access & Conference Passes.