Big Data // Software Platforms
09:56 AM
Connect Directly
Repost This

10 Big Data Migration Mistakes

Beware these pitfalls and risks when transferring your data to new computer systems or storage formats.

Transferring data between computer systems or storage formats is never a trivial task, particularly when it involves both structured and unstructured data. The complexity of data-migration jobs means that cost overruns and delays with "go-lives" are all too common, said Arvind Singh, co-founder and CEO of Utopia, a Chicago-based enterprise data solutions provider.

In a phone interview with InformationWeek, Singh outlined 10 common data-migration problems--five pitfalls and five risks--that enterprises should strive to avoid.

Pitfall #1: Failing to engage the lines of business and business users at the outset.
When companies integrate or consolidate multiple systems into one--often after a business merger--they need to identify the right business uses at the outset.

"You need to identify who knows and understands the business data," said Singh. "Who's the subject matter expert in your business? It's certainly not IT or the systems integrator."

In other words, bring the people who'll be using the data into the migration project. After all, they'll be the ones operating the system once it goes live.

2. Absence of data governance policies and organizational structure.
"You've got data being moved from System A to System B, but who owns the governance structure? Who has the rights to create, approve, edit, or remove data from the system?" Singh asked.

[ Now that you have all this data, what do you do with it? Learn 5 Ways To Benefit From Big Data. ]

Other issues that must be resolved: Is your organization set up to manage data? Is there a business process for managing the lifecycle of data? And do you have data stewards in the company?

Pitfall #3: Poor data quality in a legacy system.
Companies often don't realize that an "as-is assessment" is essential before embarking on a data-migration job.

"Understanding the quality of existing data in a legacy system is a huge pitfall that companies often don't spend enough time on," said Singh

Questions to consider: Will the existing data support new users? What is it missing? And what are you planning to do, analysis-wise, that you're not able to do today?

A detailed assessment makes it easier for companies to estimate the amount of work required to migrate legacy data successfully.

Pitfall #4: Neglecting to validate and redefine business rules.
Your company's business and validation rules may not be current.

"It's amazing how little time companies have spent agreeing on a business rule, much less making sure the data complies with the business rule," said Singh. "In other words, you think you have a business rule, but does your existing data match, map, or comply with that rule?"

In addition, auditors need to be sure that data moved from a legacy system to a new system has been validated, especially when a migration involves critical information such as financial, inventory, and payroll data.

Pitfall #5: Failure to validate and test the data-migration process.
Don't save this step for the end. "You really need to make sure that you're validating and testing throughout the process," Singh said.

Questions to consider: How are you going to test the data? Who will test and evaluate it? Who will sign off on it? And who's the ultimate consumer of the data?

This process must be built into the project's lifecycle, but unfortunately companies often "don't spend enough time aligning the data testing, validation, and migration cycles to the project timeline," said Singh.

1 of 2
Comment  | 
Print  | 
More Insights
In A Fever For Big Data
In A Fever For Big Data
Healthcare orgs are relentlessly accumulating data, and a growing array of tools are becoming available to manage it.
Register for InformationWeek Newsletters
White Papers
Current Issue
InformationWeek Elite 100 - 2014
Our InformationWeek Elite 100 issue -- our 26th ranking of technology innovators -- shines a spotlight on businesses that are succeeding because of their digital strategies. We take a close at look at the top five companies in this year's ranking and the eight winners of our Business Innovation awards, and offer 20 great ideas that you can use in your company. We also provide a ranked list of our Elite 100 innovators.
Twitter Feed
Audio Interviews
Archived Audio Interviews
GE is a leader in combining connected devices and advanced analytics in pursuit of practical goals like less downtime, lower operating costs, and higher throughput. At GIO Power & Water, CIO Jim Fowler is part of the team exploring how to apply these techniques to some of the world's essential infrastructure, from power plants to water treatment systems. Join us, and bring your questions, as we talk about what's ahead.