12 Types Of Data IT Can't Afford To Overlook
Regardless of how much data your company has -- and how much your business leaders are asking for -- you're likely missing some hidden gems. Here are 12 examples of what you might be overlooking, and why it matters to IT professionals, and to your business at large.
![](https://eu-images.contentstack.com/v3/assets/blt69509c9116440be8/blta65ee19d1cd6ab74/64cb39b39fb6672eff7e4358/hidden-data_iStock_80634505.jpg?width=700&auto=webp&quality=80&disable=upscale)
Your organization probably already has more data than it knows what to do with. Yet, it's quite likely you're overlooking, disregarding, unaware of, or unable to access important information that could directly affect analyses and business outcomes.
It doesn't matter what your universe of data is -- enterprise data or a combination of internal and external data sources -- important nuggets of information may be missing.
"Companies are collecting more data, but often struggle with what to do with it," said Dave Hartman, president and founder of technology advisory firm Hartman Executive Advisors. "Data can be extremely overwhelming in its raw form."
[What's hot in data science? Read 10 Programming Languages and Tools Data Scientists Use Now.]
Data silos are one culprit causing blind spots, which result in missed opportunities and can expose your company to risk. Today's companies need cross-functional views of their data in order to serve customers and to compete effectively.
"The untapped gems lie in the cross-links between data sources that relate to customers across channels and derive future value. This includes creating consistent and comprehensive customer profiles across the board," said Dominik Dahlem, lead data scientist at customer intelligence cloud platform vendor Boxever.
Achieving that kind of visibility isn't easy, since it requires organizations to tie together enterprise systems, which can be difficult to do despite the number of APIs available. Businesses also have to consider external data sources, such as social media or weather data, which help provide a complete view of customer behavior or serve to identify overlooked issues that might negatively affect business performance.
Constellation Research vice president and principal analyst Doug Henschen said unused data tends to fall into two broad categories. These are data exhaust and dark data, both of which have been aided by NoSQL technologies. Data exhaust is information an organization is currently not saving. Dark data is information companies may be saving but not exploring.
"Historically, companies used to throw away historical data simply because they couldn't afford to retain it. So they would keep, say, one or two years' worth of data," said Henschen. "Most people keep durable goods, such as beds and furniture, for at least five years. So targeted promotions based on two-year-old buying information may not [enable] deep enough analysis."
Dark data includes semi-structured and unstructured data that are hard to compute against. Although it's been possible to retrieve the metadata, deciphering the meaning has been a one-time, manual task. With modern search, natural language processing, semantic analysis, sentiment analysis, and machine learning, such data is now becoming computable, according to Henschen.
"The text in a CRM comment field can be analyzed and segmented across the entire customer base, rather than just looking at sentiment, one customer at a time," said Henschen. "The same goes for social network comments about your brand and products."
One way to identify missing gaps is to focus on the key metrics the organization wants to improve, which may turn out to be different than the current KPIs. For example, if the goal is to improve customer satisfaction, it's important to understand the cause of customer dissatisfaction, as well as its symptoms.
"The drivers of lower customer satisfaction might be inadequate product features and the symptoms would be lower customer adoption or product sales," said Sanjay Sidwani, SVP of marketing analytics at financial services company Synchrony Financial. "Getting a deep understanding of the drivers requires persistence in analyzing data and slicing it in multiple ways."
What data is your company missing? Once you've reviewed our 12 examples, tell us about your own experiences with hidden data in the comments section below.
[Editor's note: On page 3, the reference to Hartman Executive Advisors was corrected. Also, the quote on page 13 was updated to correct its attribution. It was said by Scott Masker, business systems engineer at MacLean Fogg.]
Enterprises struggle with the quality of customer data within their own walls and among channels. Essentially, there's no single view of the customer, because customer IDs are locked in applications and on different third-party sites. According to a recent Boxever survey of 500 customers of travel services, 70% of Millennials polled said they prefer to receive travel offers from brands via email, 19% prefer mobile offers, and 32% prefer social media offers.
However, Millennials, Gen Xers, Boomers, and even forward-thinking members of the Eisenhower generation are interacting with travel brands across multiple channels, and the way they're engaging with brands is constantly shifting. While the study is specific to the travel industry, it's not hard to imagine how it applies to other sectors.
"The question often becomes where to start, and it can be overwhelming to unpackage the myriad ways data can be interpreted to unleash its potential," said Dominik Dahlem of Boxever
Many businesses have difficulty keeping pace with their evolving customer expectations because their corporate cultures and technological capabilities can't keep up. As organizations add more types of data to the mix in order to stay relevant, they become overwhelmed.
CRM data can benefit several departments in an organization, including sales, marketing, product development, finance, and, of course, customer service. However, not all organizations are using the data in cross-functional ways yet, because there are disagreements about who should own the data.
Enterprises archive email for regulatory purposes, e-discovery readiness, internal investigations, and internal knowledge preservation. In the past, emails were discarded due to the high cost of storage. But, with falling prices and the general move toward digitization, archiving has become easier to do and more affordable than ever before.
"As more organizations grapple with data overload, IT departments are realizing they can 'reinvent' themselves to offer analytics on the knowledge and information contained in the raw data," said Greg Arnette, founder and CTO of cloud-based email archiving and analytics vendor Sonian.
Email data can be used in ways that may not be obvious, such as discovering the shadow organization by analyzing communication patterns, catching communication policy violations as they occur, and observing behavioral changes that might signal job dissatisfaction, or actual reactions to company announcements and events, Arnette said.
Imagine what's buried in instant messages, Microsoft Office documents, and the unstructured text in business applications. Yet, business processes in established companies tend to be formulated around traditional business systems. Therefore, those business processes may not account for the role played by unstructured data, which makes it difficult to ascertain the value of the unstructured information. Because governance has also focused on structured data, it can serve as a barrier to the curation, classification, and use of unstructured data.
"Analysts and organizations have been so hyper-focused on the BI and analytics associated with gleaning value from transactional data stores that they have overlooked or underestimated the value of unstructured data," said Robert Ryan, chief innovation officer at global information and communication technology company Fujitsu America. "Business decisions and most business processes rely heavily on unstructured information."
Companies that harness unstructured data can do things other organizations can't, such as analyzing patterns in business correspondence to understand real-world business processes, identify functional gaps in enterprise systems, or quantify the full end-to-end cost of goods sold. Similarly, by mining unstructured open text stored in field service work orders, companies could identify abnormal maintenance cycles and defective parts, and thereby improve maintenance processes and understand causality, Ryan said.
Today's marketers often talk about "omnichannel experiences." Omnichannel experiences transcend channels and provide a seamless user experience across devices. The concept can be as narrow as providing a TV viewing experience which follows the user from a TV to a tablet to a smartphone. More common are omnichannel experiences that flow as part of a process, such as booking an airline flight or purchasing an item.
"Often, there is an expectation that mobile replaces web interaction, yet customers do not tend to choose one channel over another. Instead, the role of mobile may actually be more exploratory in nature, where customers tend to browse on mobile devices before booking through more traditional channels like the web," said Dominik Dahlem of Boxever. "Therefore, in order to leverage the data most effectively, omnichannel interactions with customers need to be analyzed holistically."
Location-based data can help improve the effectiveness of online and offline sales and marketing efforts because it adds a layer of relevance that can help boost conversions and sales.
"Location data gives you another layer of understanding of the customer and allows companies to target individuals with personalized messages in real time," said Dominik Dahlem of Boxever. "Geolocation also allows companies to see whether targeted offers actually bring people into stores, and results in more sales."
Increasingly, brick-and-mortar stores are adding more digital elements to their stores and their brands to stay relevant and track customer behavior in ways not previously possible or practical. More data points mean more correlations and insights to provide an accurate view of customer behavior.
Ratings and reviews are table stakes for companies with an e-commerce presence these days. The reviews are there for all the world to see, and yet companies aren't necessarily leveraging the data to the best of their abilities. Conceptually, they understand purchase considerations and conversions are driven by consumer-generated content (CGC), but they're not yet using the data to inform or shape marketing campaigns.
"Ninety-two percent of consumers trust peer recommendations over branded content," said David Moon, VP of strategic consulting at ratings and review platform vendor Bazaarvoice. "The biggest reason companies fail to fully leverage CGC is the misconception that CGC is a web-only asset."
According to Aleksander Kijet, VP of operations at data science anti-fraud and business intelligence startup Nethone, females are 8% more likely to carry out an authorized transaction than males. Yet, for some verticals in which gender weighs heavier, the opposite can be true. In other words, gender is a supportive clue, rather than a decisive factor.
"[F]raud prevention must be based on thorough, in-depth analysis of a multitude of parameters, which is possible only through the use of advanced machine learning techniques powered by extensive data mining and data enrichment," said Kijet. "Gender is one of the apparently irrelevant variables that, in certain circumstances, might be of significant value, but only when accompanied by a multitude of other variables and with appropriate analytical tools applied.
Big brand retailers are doing all sorts of things to help quantify and track customer behavior in brick-and-mortar stores, including installing kiosks and beacons, leveraging store security video in new ways, and -- in extreme cases -- testing smart mirrors in dressing rooms. However, such technologically advanced stores are still in the minority.
"What's really happening is [data] only gets captured at the register -- and not all of it. We see the results, but not the choices not made, options rejected, and no view of the inputs that caused the outcome that actually happened," said John Parkinson, affiliate partner at management consulting firm Waterstone Management Group. "It's easy to build false predictions based on what we do see that [fail to] consider what we don't see."
Analyzing entire populations can be more expensive and time-consuming than is practical, which is why sampling techniques have been necessary. Despite the availability of modern technologies such as Hadoop, some industry observers argue analyzing entire populations still isn't necessary. Others caution against this line of thinking.
"Many organizations based major decisions on sampled data, but this leaves room for significant error. With the tools readily available, there's no excuse not to be looking at populations of data," said Nate Smith, product marketing manager at Adobe Analytics. "We see brands not leveraging mobile to its full capability and not truly understanding the big questions. How is mobile engagement happening? Should we build an app? How is location and/or time affecting the types of decisions people are making?"
Healthcare providers are transitioning from paper-based processes to digital processes. The move has resulted in the Electronic Healthcare Record (EHR). Doctors and clinicians do their best to diagnose and treat patients based on the information they have, which is usually what is reported by the patient and perhaps other healthcare providers who have treated the patient.
However, they have little or no insight into the daily lives of their patients, such as how much they drink or smoke, how much or how little they eat, what kinds of foods they eat, whether they exercise or not, etc. While there are privacy concerns about connecting such dots, the additional information could enable improved accuracy in diagnoses and treatment.
"Lifestyle and habits play a tremendous role in a person's health status. Income, demographics, location, and buying habits can all provide a more complete picture of a healthy consumer than clinical data alone," said Brian Garcia, CTO and SVP at health optimization platform vendor Welltok. "This consumer data is critical in understanding an individual's needs, and how their heath can fit into their life, as opposed to forcing individuals to fit in a static healthcare system."
EHRs were not designed to include all that data, however. Many of the systems are designed for regulatory compliance and billing, not patient care, Garcia said.
-
About the Author(s)
You May Also Like