How to Buy External Data to Fuel Analytics, AI Insights

Your enterprise organization probably has set up data management and data analytics operations to get the most out of the data generated by your organization. Chances are you are also at least exploring or piloting machine learning as part of your data operations. But is your data telling you the whole story?

Sometimes incorporating a new data stream from outside the organization can add a whole new level of insight and value to your analytics or machine learning efforts. A Gartner survey of 196 organizations released in February 2018 revealed that 46% of organizations are using external data sources.

The most common example of this is adding a stream of weather data to the mix to help retailers predict demand for certain products at certain times -- snow shovels or sun screen -- or to help insurance companies inform their end customers about preparatory actions to take to prevent property damage from an impending storm, like moving their cars to higher ground ahead of predicted flooding.

But buying data is still a relatively new practice. Sure, organizations are doing it on a regular basis. But there's no giant trusted provider or marketplace of vetted quality data sets and data streams.

Alternatively, if an organization recognizes it has a data stream that's of value to sell on the market, there's no single clearinghouse to go to that will vouch for your data quality and help you find buyers.

"There's a lot of dynamism in the data provider marketplace," said David Schatsky, a managing director at Deloitte Consulting who specializes in analyzing emerging technology and business trends. "There are lots of players starting up. There are new specialist providers."

David Schatsky

Schatsky is the author of a new report, Data ecosystems: How third-party information can enhance data analytics, that provides a sense of where industry is in terms of data being bought and sold and he offered some guidelines to help organizations to navigate the sometimes tricky practice of buying and selling data.

He encourages enterprise organizations to develop a new core competency around the practice of buying and selling data. It starts with scouting data sources that can provide an individual business or organization with an edge.

"This is a bit of a process," he said. "Just as some companies scout for new technology, they will also be scouting for new data sources."

Once a new source is identified, organizations need to test a sample of the data for quality and for alignment with other data they may already have in house.

Organizations must also be cognizant of any regulations around the data they are buying or selling. If data was generated by your customer, who has the rights to that data? It may be important for the data team to have access to legal experts to sort out any rules or restrictions around data they are buying or selling.

If you are buying data, you may also want to consider whether you want to buy just the data or if you want to buy insights from the data. The Deloitte report points out that data brokers collect data from multiple sources and offer it in collected and conditioned form. However, you can also buy data that's already been enhanced by applying analytical rules and calculations, such as scoring or tagging of objects. In addition, there are also services where organizations can submit their own data with specific analytical requests, and providers will combine that data with data from other sources to provide insights.

The team evaluating and managing the external data process should be led by the chief data officer and have links to business, IT, and legal teams, according to Schatsky. Companies should also "think of themselves as participants in a data ecosystem, which some have defined as a network of actors that directly or indirectly consume, produce, or provide data and other related resources."

Now's the right time to put together this kind of core competency team for data scouting and external data relationship management, especially as companies look to create machine learning and AI practices.

"There's so much innovation. The technology that companies are using gets a lot of attention. But AI technology requires data to run on," he said. "This is an important part of companies' innovation strategies."