Rise of the Data Disruption Economy in the Wake of COVID-19

New data strategies required to meet demands imposed by the pandemic may also be signs of seismic shifts in the market, some experts say. On Tuesday, Raj Verma, Co-CEO of MemSQL, a SQL database management system, held an online fireside chat with Dave Menninger, senior vice president and research director of data and analytics research from Ventana Research. They discussed reshaping architecture in response to COVID-19 and touched on how Snowflake going public, in what is touted as the biggest software IPO to date raising $3.4 billion on a $33 billion valuation, might indicate a tangible shift in this space.

“Is there a changing of the guard in the data industry?” Verma asked. “And what’s driving it?”

The catalyst for this can be traced back to a key source, according to Menninger. “The cloud has fundamentally changed the way we do business,” he said. The cloud enables business processes to continue remotely during the pandemic, Menninger said, while also offering opportunities for platform shifts and massive disruption. “The momentum, and the interest, is with these newer organizations,” he said, citing Snowflake. “It’s not that older organizations [vendors] have gone away, but the excitement is around newer ways data is managed and dealt with.”

Such excitement and momentum, Verma said, could push the database market cap to $1 trillion by 2025. “We already are at about $400 billion, $500 billion,” he said, pointing to the worth of Amazon, Microsoft, and Google BigQuery in this segment. Verma also said the pandemic accelerated the need to advance data strategies and projects, in some instances by years.

There was already an ongoing trend to embrace the cloud before COVID-19, Menninger said, with that trend projected to continue. “In our research, 86% of organizations expect the majority of their data to be in the cloud,” he said. “Doesn’t mean it’s in the cloud today, but they expect the majority of their data will be in the cloud at some point in the future.” About 99% of organizations, Menninger said, expect to do analytics in the cloud and half of them are already doing so. Further, one-third of all organizations’ primary datalink platforms are in the cloud, he said.

The pandemic catalyzed and accelerated those trends, Menninger said, with some organizations caught flatfooted and trying to catch up. “If you were not embracing the cloud in some way, shape, or form, you’ve got a lot more complicated configuration and connection to your employees, who are now all remote,” he said. “Imagine if you didn’t have a VPN set up beforehand. You’ve got a lot of work to do.”

Dave Menninger, Ventana Research

Given the hodgepodge of approaches to cloud organizations might pursue, Verma asked whether many ecosystems will be hybrid for the foreseeable future. Menninger said his firm asked organizations what they do with their data processing, what systems they take data from and where they put it. Despite the buzz that surrounds the cloud, he said, two-thirds of data movement is on-premises to on-premises. “Another a quarter were going one way or the other: cloud to on-premises or on-premises to cloud, and about 15% were doing cloud-to-cloud,” Menninger said. Pure cloud remains a small fraction of the market, he said, and a little less than half were doing some sort of hybrid configuration. “I don’t expect that to go away any time soon.”

In many instances, the introduction of the cloud is not to replace existing systems, Menninger said. Organizations look to the cloud to implement greenfield opportunities, he said, and to explore what they could do with the cloud and which existing systems might be migrated to the cloud. Financial and healthcare organizations naturally may have policies or regulations that restrict from using cloud for certain data and systems, he added.

The rippling effects of the cloud has disrupted architecture even for on-prem, he said, with organizations considering running containerized systems as opposed to historical systems such as massive parallel processing. “Part of the value of this disruption is that it’s forcing us to rearchitect our systems, and to architect them in ways that provide elasticity, scalability, economies of scale, and agility,” Menninger said. Cloud architecture could inspire organizations to adapt and uncover new means to compete in these shaky times. “We have to be agile because of all that uncertainty,” he said. “The Dow may say one thing but who knows what’s going to happen tomorrow.”

For more content on cloud migration, follow up with these stories:

Cockroach Labs, Neo4j, Snowflake Talk Infrastructure Future

The Growing Security Priority for DevOps and Cloud Migration

Next Steps for Cloud Infrastructure Beyond the Pandemic

Cloud Scalability Shows Its Worth in the Wake of COVID-19