Data Governance in the Age of Exponential Technology

An important new source of value is emerging in the enterprise: the data associated with the so-called exponential technologies, exemplified by the Internet of Things (IoT), augmented reality (AR), artificial intelligence (AI) and robotics.

These technologies generate and consume huge amounts of data, which, with proper governance, can become the foundation of new and disruptive modes of customer engagement, new products and services, new business models, and digital transformation in general. While each presents its own unique set of opportunities and challenges, there are two areas of data governance that are critical to all of them: security, and the need for holistic, enterprise-wide governance, as opposed to governance on a silo-by-silo basis.

Bridging the trust gap

Relationships between companies and their customers and stakeholders have always been based on mutual trust, but in recent times many companies haven’t kept up their end of the bargain. Regulations like GDPR and the new California Consumer Privacy Act are symptoms of the chasm of trust that’s been created by endless data breaches and the use of data beyond what customers intended or authorized. 

To restore trust, data governance based only on regulatory compliance isn’t enough. Companies must become true custodians of data, and data governance must be driven by customer and stakeholder needs. Given the complex and fragmented world of data in today’s enterprises, this requires a broad, end-to-end strategy.

A holistic view

Everyone who touches data is familiar with the strong tendency toward data fragmentation. For instance, the data that’s ideal for procurement may not meet the needs of the factory floor, much less those of the warehousing system or of retail outlets, even though there’s a great deal of mutual interdependence among these business functions.

The exponential technologies are no exception to this tendency toward data silos. Early iterations of these technologies may be viewed more as pilots or experimentation, so the priority to integrate and share data and metadata from other parts of the business may be minimal. But at an architectural level, the integration and governance of all the necessary data that feeds these technologies must be a core design principle, not an afterthought. Otherwise, the process of moving from experiment to reality will be significantly delayed.

The Internet of Things

With IoT, the data governance challenge relates to all of the “3 Vs –  volume, velocity, and variety. Companies are still learning how to deal with big data, and IoT will eclipse big data volumes by orders of magnitude. Furthermore, IoT produces streaming data, which is very different from traditional transactional data. If companies are to take advantage of this data in real time, they’ll need to learn how to parse and process streaming events on the fly,  often with no control over the sources of that data or how it’s collected. Companies that can achieve this will be able to offer compelling features to their customers. For example, a connected car monitored by the manufacturer could inform the driver of low oil or a problem with a sensor before any damage occurred.

Ingesting, filtering, and aggregating all this data requires careful planning, and “small” details like frequency of sensor readings can play a big role in success. Storage is another issue. The current approach of batch capture is not viable on a long-term basis because there will simply be too much data for any company to store. Finally, ways must be found to integrate this data with other enterprise data, including data residing in legacy systems, to truly generate value from the new insights this data can illuminate.

Augmented reality

Augmented reality relies on two types of data: positional data (location, orientation) and the corresponding assets, both textual and visual, that are needed to augment what the user is seeing. Good governance in this area means making the required assets available instantly, a need that may affect storage options. Some assets, such as those associated with retail, will also need to be regularly updated, but the experiences AR can provide are worth the effort. For example, customers entering a department store can be guided to specific displays based on their purchasing history, and once they arrive they can view themselves in AR-equipped “magic mirrors” wearing the latest fashions without having to physically put on the clothes.

Artificial intelligence and robotics

Data governance is a key factor in the success of AI initiatives, because without available, relevant and trusted data, there is no AI. The reliability and value of any AI capability is directly dependent on the data available to it. This fact makes data management functions like discovery and mastering more important than ever. Data scientists developing algorithms must easily find the data they need, no matter where it is located or in what form. They also need to trust the quality of the data they’re working with.

Data quality is critical with machine learning, given that large quantities of trustworthy data are required to properly train an ML system. Data that is of poor quality or simply incomplete leads to problems that are sometimes serious. To take an extreme example, ML-based X-ray analysis for detecting pneumonia or cancer based on inadequate data could put lives at risk by failing to detect disease.

Clearly AI and ML are also foundational technologies for robotics because they enable robots to learn how to correctly perform the tasks for which they’re designed, such as grasping an unfamiliar object.

With both AI and ML, it’s important to remember that the algorithms not only consume data. They also create data, and some of it will be used to feed other algorithms. For this reason, data governance will likely evolve over time so that it includes algorithmic governance.

Intelligent disruption

Data from the exponential technologies will clearly be a disruptive force. A striking example is the new Series 4 Apple Watch, which can take an FDA-approved electrocardiogram. This capability will result in a much richer data set for any given patient, as it can record the heart’s behavior under a variety of conditions that might be difficult to simulate in a clinic, e.g. while exercising, under emotional stress or sleeping. It will also reduce the cost of the procedure by eliminating the need to visit a clinic. This could be viewed as a disintermediation, where the clinic is the middleman between the doctor and the patient.

This is only one of many, many examples of how exponential technology data can turn established processes upside down. With holistic data governance based on the needs of users and stakeholders, IoT, AR, AI and robotics will all generate data with disruptive potential in the very near future.

Amit Walia is president of Products and Strategic Ecosystems for Informatica.