Moving Data and its Hidden Challenges

When working with vendors on data ingestion, CIOs and IT teams sometimes fail to take complicating factors into consideration. Here’s how you can navigate.

Guest Commentary, Guest Commentary

May 22, 2019

5 Min Read

“Yes, I’d like a moving quote, please.”

“Of course. Glad to help. How big is your house in terms of square feet?”

“Not sure.”

“No problem. How many bedrooms?”

“Maybe four?”

“Do you have a finished basement?”

“I believe so.”

“Is there an attic with storage?”

“Yes, but not sure what’s up there.”

“Would you like us to pack for you?”

“Please do. How much will this cost?”

“Well, we’ll have to come out to your house, determine how much stuff you have, figure out the safest and most efficient means of getting it in the truck, and then calculate distance. Where is this all going to anyway?”

“TBD. It’s not built yet. Anyway, how much will it be, and can we have it all moved very soon? We’re kind of in a hurry.”

This exchange, while humorous, is not unlike conversations we have with chief information officers and IT teams every day when discussing data ingestion. To get data from a multitude of sources into a single repository where everyone in your organization can access it and benefit from it (with appropriate permissions and safeguards, of course) is an exacting science. You can’t just spitball it most of the time because there are simply too many options and the consequence range can be wide. Everyone gets you’ve got a new house for your data and it’s exciting, but the time and money questions depend on the shape your data’s in and how ‘move-ready’ it appears.

'It’s just some transaction data'

This is a common refrain heard by vendors. In fact, we have one engagement and loyalty platform client with 1,000 unique fields for each transaction. We need just 50 of those to work our magic. Mapping which of those 1,000 correspond to which of our 50 is painstaking and arduous. Now imagine that you want a single 360-degree view of each of your customers, but you have four or five different point-of-sale systems pushing transactions into our platforms (common in many restaurant and retail companies with franchisee groups, stores of varying ages, different brands purchased through acquisition with individual systems, and so on).

There are no cookie-cutter transactions. The very definition of a transaction, what’s included, and how it’s constructed varies widely from vertical to vertical, company to company, and even within companies. It’s a job that must be done right, so shortcuts need not apply.

We are family

Most households have multiple members. Oftentimes companies get requests to roll up data, like loyalty points earned: for example, by anyone in the family into a single balance that can be spent by anyone in the family.

Seems straight-forward, right? Not really. Is there an alpha member who serves as the admin for the household? Can everyone not only see the balance, but use it at their discretion? Do members keep their own balances or is everything communal? If so, can they be transferred from one member to another? Can they be ‘gifted’ outside the household?

Seemingly subtle distinctions like this point to the non-technical challenges with seemingly simple aspects common to most marketing and loyalty programs. Decisions must be made prior to implementation, and they have cascading effects that must be thought through all the way to the end.

Product catalogs

You have a set of products you sell, right? Usually, wrong. You have sets of products, plural: online, offline, regional, concept stores, and so forth. Determining who is eligible to buy what and where is a surprisingly scientific proposition. Even an absolute paragon of consistency like Starbucks has subtle variations in offerings. Imagine a grocer with different SKUs depending on location footprint. Some might have floral. Others don’t have the footage for it. Regional differences have impacts as well. A Wegman’s in Buffalo, New York, may have five variations of kimmelweck rolls, catering to the market’s local favorite. The Wegman’s in Chestnut Hill, Massachusetts, might have none.

While there are many more commonalities than exceptions in product offerings, it’s those exceptions that can draw the ire of the customer. That’s the exact opposite of what you’re trying to accomplish. Take your time, think it through, and you’ll have something for everyone. What that “something” is may vary, however.

Get the foundational elements in order

Getting the data right is the absolute foundation to all data-driven initiatives, whether loyalty orchestration and customer interaction management as outlined here, or any new AI or machine learning implementations your team and your vendors are working on. So, it’s worth taking the time to make sure that foundational element is letter-perfect. How much time that takes depends on how much data there is and the condition it’s in. If your data house is boxed, labeled, and waiting for the truck, it shouldn’t take long at all.

If everyone’s not on the same page whether it’s a couch, loveseat, or davenport that gets moved next, it could take a while longer. We don’t all have the same frame of reference. Things are called different things at different places among people of different levels of experience and background. The best advice is to prioritize what’s most important, take this opportunity to archive or get rid of what’s no longer actionable moving forward, and get both a schema and nomenclature that’s universally understood.

Patrick Reynolds is CMO at SessionM, a customer engagement and loyalty platform empowering innovative brands to forge stronger and more profitable customer relationships. The platform serves brands including Chipotle, Coca-Cola, L’Oreal, and Kimberly-Clark, and scales for the enterprise, globally. Prior to SessionM, Reynolds ran marketing and strategy for two successful startups in the streaming audio industry. He also has client-side experience as CMO of a publicly traded retailer and has held multiple leadership positions with leading international advertising agencies. 

Read more about:


About the Author(s)

Guest Commentary

Guest Commentary

The InformationWeek community brings together IT practitioners and industry experts with IT advice, education, and opinions. We strive to highlight technology executives and subject matter experts and use their knowledge and experiences to help our audience of IT professionals in a meaningful way. We publish Guest Commentaries from IT practitioners, industry analysts, technology evangelists, and researchers in the field. We are focusing on four main topics: cloud computing; DevOps; data and analytics; and IT leadership and career development. We aim to offer objective, practical advice to our audience on those topics from people who have deep experience in these topics and know the ropes. Guest Commentaries must be vendor neutral. We don't publish articles that promote the writer's company or product.

Never Miss a Beat: Get a snapshot of the issues affecting the IT industry straight to your inbox.

You May Also Like

More Insights