Why Now is the Time to Consider Data Consolidation

As data flows into enterprises from a seemingly endless array of sources, IT leaders face the challenge of delivering relevant data to the parties who need it most, including financial analysts, planners, and sales teams.

Consolidation streamlines daily operations by gathering data from disparate sources and storing it in a central location. The goal is to improve data quality, increase data security, and make life easier for data users. Data consolidation improves data management, combining information from diverse sources into a unified and comprehensive perspective, says Shixiang (Woody) Zhu, an assistant professor at Carnegie Mellon University’s Heinz College of Information Systems and Public Policy, in an email interview. “This significantly enhances data integrity, aids in making well-informed choices, optimizes data administration, and bolsters data protection among other benefits.”

Data consolidation is needed to establish trust in decision making, observes Raymond Velez, CTO of customer data at digital consultancy Publicis Sapient via email. “Increasingly, as more regulatory requirements are imposed on large organizations, data consolidation and governance can ensure teams are avoiding penalties and treating consumer data safely.” Without data consolidation, he notes, multiple and duplicative data sources spread across organizations can make getting to a single source of truth difficult.

Related:Unlock a Galaxy of Opportunity With Constellation Analytics

Growing cloud adoption is also driving the need for data consolidation. “With the cloud, you have more data from more sources that can be used to generate insights, fuel impactful business decisions, or enable the use of emerging technologies,” says Salim Syed, vice president and head of slingshot engineering at Capital One Software in an email response. With so much data pouring in from various cloud sources, achieving speed and scale can become challenging without consolidation.

Multiple Benefits

Data consolidation’s benefits include greater trust and transparency, regulatory compliance, and increased efficiencies. “With data consolidation programs in place, organizations can avoid duplicative efforts while improving data quality,” Velez says.

When approached correctly, data consolidation makes it easier for end users to find, access, and use data by presenting it in a unified view, Syed says. “Data consumers are then able to leverage high-quality, well-governed data quickly, at scale, to make strategic decisions.”

Adopting data consolidation requires careful planning, starting with a clear definition of objectives and a comprehensive inventory of data sources. “Cleaning and standardizing data is crucial for consistency across different datasets, necessitating the selection of suitable integration tools and technology that ensure scalability and performance,” Zhu says. User training is also essential to maximize consolidation benefits, he adds. To ensure the long-term consolidation success, Zhu suggests conducting a post-implementation review and committing to continuous improvement.

Related:How to Safely Delete Obsolete Data

Getting to Work

Data consolidation requires careful planning, says Kathy Rudy, chief data and analytics officer with technology research and advisory firm ISG in an email interview. “The best and most important reason to consolidate data sources is to provide your organization with better, new, and more significant insights,” she explains. “If consolidation will drive more meaningful insights and create additional value out of disparate data sources, it will become a high priority task.”

Data consolidation is typically based on extract, transformation, and load (ETL) approaches. Such methods allow organizations to modernize their data management with cloud-based approaches that leverage ETL to guide data egress and ingestion across multiple layers of adaptation, including traditional data warehouses as well as modern data lakes.

Related:7 Steps to a Data-Centric Nirvana

Rudy says it’s important to create a collaborative environment between IT and business leaders. “The business typically owns the data, and IT typically is tasked with consolidation,” she notes. Without business input on how the data is used, how it could be more valuable in a consolidated state, and an overall acknowledgment that the activity will provide value, it will be a hard road to travel. “From a pragmatic perspective, consider how to approach data quality, security considerations, technology requirements and any organizational changes as an outcome of consolidation.”

Potential Pitfalls

One of the biggest data consolidation mistakes is neglecting data quality and consistency across different datasets, Zhu says. “This mistake can occur when there’s insufficient attention given to cleaning and preparing the data before consolidation.”

One of the most overlooked needs in data consolidation is strong governance, Velez says. “Building an organization with a variety of defined roles and needs is critical to data consolidation,” he explains. If not handled correctly, the data consolidation effort will miss the mark.

Final Thought

It’s never too early to begin consolidating data, Syed observes. “It’s much easier to consolidate data from the beginning, so the goal should be creating mechanisms to organize data early on, such as at the beginning of a cloud migration or when bringing new data into broader enterprise systems.”