If 20 billion devices are projected to be connected to the Internet by 2020, the volume of data available will be on a scale never experienced before. That will open opportunities to exploit data and gain insights that IT and business managers haven't had access to previously.
Among other things, the Internet of Things is going to challenge current data capture, storage, and analysis practices. It will move data analysis out toward the edge of the network, closer to where the data is being generated. It is also likely to require new types of database technology.
Those were a few of the conclusions in a 14-page report, "Harnessing Data in the Internet of Things," by Gary Barnett, chief software analyst at market research firm Ovum. If he's correct, the volume of data generated by the IoT will require IT managers to rethink how they collect, manage, and store data on multiple fronts. Failure to do so, he concluded, will amount to a significant loss of competitive advantage.
The Internet of Things poses challenges not only in the volume of data generated, but also in gathering that data in a cost-effective way when network bandwidth may be at a premium. Barnett suggests that much preprocessing of data will need to be done close to its point of origin.
Aggregating 100 sensor readings into one reportable average would allow a central system to get the feedback it needs without choking the network.
"Managing data by collecting it at the edge and centrally storing and processing it will be inappropriate to many IoT networks ... Data management will be pushed out from the center into the network itself and onto the devices that sit at the edge," Barnett wrote.
But sensors and other devices will run on a variety of energy-saving processors, not necessarily the complex and energy-hungry x86 architecture, and the amounts of memory available will shrink.
"Some IoT projects are being built around processors that have as little as 512 bytes of RAM," Barnett noted.
Distributed databases that can be embedded in devices to sort and aggregate data will need to be able to run on a variety of hardware architectures with a variety of interfaces.
These distributed systems will also need to be able to handle both unstructured and structured data without taking up much memory. They will be called on to capture geospatial, time-series, and unstructured behavioral data off the systems they are monitoring, filter and aggregate it, and send it in compressed form to a central analysis system. There may be many "gateway" database systems near the edge of the network performing this function for a single, powerful central system.
These distributed systems will need to have characteristics that have only recently become part of the central enterprise system, including the ability to be self-tuning and self-healing, with automatic recovery from failure, if one occurs.
The competitive value of IoT data will come as it can be collected, filtered, and made available for analysis in near real-time. Being able to forecast events and set triggers that respond to the events captured in near real-time data will make the Internet of Things more useful than we can currently envision.
GE, with its Predix IoT system, is building systems that can collect IoT data and assess equipment operations, detecting the signs that a failure may occur in the near future. Its operational goal is no longer to repair and recover from failures, but prevent them entirely.
"The Industrial Internet is real. It's about more efficiency, more velocity, better safety ... It's about no unplanned downtime," GE CEO Jeffrey Immelt said in an address to the GE Minds + Machines conference in San Francisco last September.
In a similar way, IoT will change how retail, healthcare, lending, travel services, and other products and services are conceived and delivered in the new digital economy.
"The dispersal of IoT data to the edge of the network means that the management of that data also needs to be dispersed," said Barnett. Coordinating the systems at the edge with a central source of intelligence will present constant ongoing challenges as the IoT evolves, he wrote.