How 2020 Impacted 2021's Predictive Modeling - InformationWeek

InformationWeek is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them.Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

IoT
IoT
Data Management
Commentary
4/13/2021
08:00 AM
Lisa Morgan
Lisa Morgan
Commentary
Connect Directly
Twitter
RSS
50%
50%

How 2020 Impacted 2021's Predictive Modeling

COVID-19 disrupted the world and predictive models in 2020. In 2021, we're somewhat better prepared to deal with extreme uncertainty.

Image: momius - stock.adobe.com
Image: momius - stock.adobe.com

Before the pandemic hit, digital disruption was the major force shaping the direction of business models and entire industries. As 2020 demonstrated, digital disruption seems relatively tame now compared to COVID-19's impacts on business. The sudden and dramatic shifts in everyday realities negatively impacted predictive model accuracy because they were so inconsistent with historical data.

"One of the really big things that people grappled with is the fact that they took for granted that the models were built properly," said Scott Zoldi, chief analytics officer at decisioning platform provider FICO. "Obviously we were in a time of huge stress so as people were trying to understand how to pivot their business, instead of asking, 'How can I leverage the asset I have?' they basically said, 'Let's just throw the model out and build a new model’, which comes with a whole set of other issues because we essentially then have models built on nonstationary data."

Adapting to the New Normal

Companies have traditionally had years of data that could be used for predictive purposes. However, when the entire world changes so radically in such a short time -- supply and demand, supply chain disruption, shuddered businesses, stay at home mandates -- it's time to get creative.

"A traditional approach would be to look at sales and understand trends. Now sales is not a good predictor so you have to look for something else," said Dan Simion, VP of AI and analytics at global consulting firm Capgemini North America.

Dan Simion, Capgemini
Dan Simion, Capgemini

For example, one of Capgemini's airline clients is using future bookings to predict business travel instead of sales because the demand for business travel evaporated in 2020.

While companies understood they needed the ability to adapt to change quickly, they also understood they needed to minimize risks by using data to make decisions.

"Trying to predict using traditional statistical techniques gets harder and harder because you need a lot of data points and observations," said Simion. "Before you would have one observation each week or each day and you could go down to every hour."

The same thing holds true for other dimensions that can be decomposed into smaller pieces -- zip codes instead of countries or regions, for example.

"That is increasing the degrees of freedom, the number of observations within the same time frame," said Simion.

Contingency Planning Is "In"

At a business strategy level, organizational leaders were warned that they needed to do contingency planning at an entirely different level than they had before. Instead of having a plan A and a plan B, global consulting organizations were advising clients to have several contingency plans covering different scenarios such as lock downs and supply chain disruptions. However, the same type of thinking didn't trickle down to the data team in many organizations.

"We're seeing a pickup in demand, especially lately," said Simion. "The question used to be, 'What is the contingency plan?' and now it's 'What are my options for shipping route if I can't ship through traditional routes? Where should I place my containers to account for that?"

Why FICO's Predictive Models Weathered the Pandemic Better Than Most

FICO had fewer challenges with its predictive models in 2020 than most other organizations. Then again, customers rely on its models to make important business decisions such as whether to issue credit and at what level.

Scott Zoldi, FICO
Scott Zoldi, FICO

"Prior to COVID, we were always criticized. Why do your models take so long to build when this Fintech over here can do it in the cloud [a lot faster]?" said Zoldi. "We would say to the client, 'You and I both depend on this model and therefore we have to understand it carefully and we have to build it carefully."

Part of FICO's secret sauce is a four-prong methodology that includes:

  1. Robust AI, which focuses on model performance and stability
  2. Explainable AI, which is about understanding relationships in a model, including what the model is learning
  3. Ethical AI, which involves testing to ensure ethical outcomes
  4. Efficient AI ,which captures information from the earliest stages to:
    • Understand the data
    • Do scenario testing
    • Decide whether the behaviors that drive the model make sense
    • Understand what to monitor

Zoldi also underscored the importance of a governance model or model development governance model.

"If you don't have a process written down and codified to establish that from this point forward, we're only going to use these technologies, have these kinds of people review the model, these are the standards for what it means to build a robust and responsible model, and out of that would come things the organization would want to monitor to make sure the model is performing properly," Zoldi said.

In a forthcoming report sponsored by FICO, 90% of the CIOs, chief data officers and chief AI officers surveyed said they have to make fundamental changes and investment in how they monitor their models.

"I think if 90% of analytics leaders in these different firms say we have a huge amount of work to do in monitoring I think that's probably one of the big things to look at in 2021," said Zoldi. "The other thing to focus on in 2021 is that if models are built properly and carefully, you don't lose their predictiveness but their interpretation changes a little bit meaning you might use a different score threshold than you did before."

Related Content:

Making Predictive Analytics Work in an Uncertain World

How IT Can Get Predictive Analytics Right

IoT and Predictive Analytics: What We’re Driving Toward

Why Everyone's Data and Analytics Strategy Just Blew Up

 

Lisa Morgan is a freelance writer who covers big data and BI for InformationWeek. She has contributed articles, reports, and other types of content to various publications and sites ranging from SD Times to the Economist Intelligent Unit. Frequent areas of coverage include ... View Full Bio
We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.
Comment  | 
Print  | 
More Insights
InformationWeek Is Getting an Upgrade!

Find out more about our plans to improve the look, functionality, and performance of the InformationWeek site in the coming months.

News
Pandemic Responses Make Room for More Data Opportunities
Jessica Davis, Senior Editor, Enterprise Apps,  5/4/2021
Slideshows
10 Things Your Artificial Intelligence Initiative Needs to Succeed
Lisa Morgan, Freelance Writer,  4/20/2021
News
Transformation, Disruption, and Gender Diversity in Tech
Joao-Pierre S. Ruth, Senior Writer,  5/6/2021
White Papers
Register for InformationWeek Newsletters
2021 State of ITOps and SecOps Report
2021 State of ITOps and SecOps Report
This new report from InformationWeek explores what we've learned over the past year, critical trends around ITOps and SecOps, and where leaders are focusing their time and efforts to support a growing digital economy. Download it today!
Video
Current Issue
Planning Your Digital Transformation Roadmap
Download this report to learn about the latest technologies and best practices or ensuring a successful transition from outdated business transformation tactics.
Slideshows
Flash Poll