Data in Motion Drives Enterprise Change

Previously, collected data remained in one place. It arose from a transaction system, went to a warehouse or mart, and remained there for users to access and analyze. Today, however, data is often in motion. It may originate in a sensor on a moving vehicle, then is aggregated, sent to the cloud, into a data lake, then out into the world to guide actions and decisions. Sometimes action is taken on this data at the edge or at the point of origination in the sensor or the device.

Contemporary business offers many settings where data is in motion, including: 

Why is more data now on the move?

There are several contributing factors to why data is increasingly in motion. The Internet of Things has taken off, and sensors on products, pallets, and machines churn out data. Mobile data and communication speeds are also both growing tremendously.

Edge AI chips are poised for extensive roll-out, with more than 750 million expected to be sold in 2020 (sales set to exceed $1.5 billion). These devices will create sensor-generated data to be stored, processed, analyzed, and acted upon close to or at the edge of networks. 5G wireless networks will eventually dominate the marketplace on speed, latency, penetration, and capacity.

On the customer front, most have smartphones, and expect to use them to interact with businesses for convenience and mobility -- especially as COVID-19 has increased the need for virtual and intelligent interactions with work, shopping, health, and entertainment.

But sensors, mobile devices, and virtual living are not the only factors driving data in motion. Organizations need new tools to consume and make sense of all the moving data, and there have been amazing innovations in analytics and AI. Together, these innovations and trends make data-in-motion seem inevitable.

Rearchitecting data for motion

Companies may need to transform and modernize their data architectures to master data in motion. The time-honored architectures that worked for non-mobile data are not well-suited to mobile data. Data in motion architectures can be complex, requiring multiple layers for analyzing and acting on data; they need multiple local area networks and a wide area network.

Most such data architectures will also include a more flexible cloud component to allow data to be stored in multiple formats and translated for integration and aggregation. Cloud architectures also address data security, customer and product identity management, data privacy regulation and 24/7 availability. Cloud vendors often offer specialized applications that may assist with particular data-in-motion use cases, such as a set of cloud services for connected vehicles. Data lakes are typically necessary for storing large volumes of IoT data -- in the cloud or on-premise -- in multiple data formats and don’t require a time-consuming ETL process.

Data in motion involves several different situations that might require different architectures and applications, including: 

The most complex architectures are when the asset and data both move. The widespread use of 5G networks -- just beginning to be rolled out in the US, but further along in Europe and Asia -- is expected to make this type of network much more feasible.

Managing data in motion

There are also multiple management issues to address, such as monitoring bandwidth availability and response time, as well as strategies for false anomalies generated by sensors. Furthermore, consumption and analysis of data may be partially automated, but there is usually a “human in the loop” who monitors and acts on results. If this human component isn’t reliable, it’s just as problematic as a faulty sensor or cloud outage.

There will need to be ongoing attention to customer data privacy and usage permission issues. Given the fact that data-in-motion issues may cross organizational boundaries, there may also be data ownership issues to resolve. For example, using jet engine data for predictive aircraft maintenance, ownership may be claimed by engine manufacturers, airframe manufacturers, airlines, and even the airline pilot association.

Get your data moving

Despite the challenges of data in motion, it’s time for organizations to get moving. You likely already have data in motion or opportunities to create it. Your employees, customers, and products, parts and equipment are all moving, and you need to know where they are and what they are doing. It’s unlikely that your current technology environment can handle those requirements. Only a strategy, an architecture, and a management plan will allow that.

Ashish Verma is the Data and Analytics Modernization Market Offering US and Global leader of Deloitte Consulting’s Analytics and Cognitive practice that enables clients to plan and execute new businesses strategies with technology innovation while driving analytics into day to day operations, evolving key data and analytics capabilities to meet changing market dynamics and deliver IT enabled business transformations.

Tom Davenport is the president’s distinguished professor of Information Technology and Management at Babson College, co-founder of the International Institute for Analytics, fellow of the MIT Initiative for the Digital Economy, and senior advisor to Deloitte AI. He has written or edited 20 books and over 250 print or digital articles for Harvard Business Review (HBR), the Financial Times, and more. He earned his Ph.D. from Harvard University and has taught at the Harvard Business School, the University of Chicago, the Tuck School of Business, Boston University, and the University of Texas at Austin.