Never Miss a Beat: Get a snapshot of the issues affecting the IT industry straight to your inbox.
Enterprises Are Racing to Leverage GenAI, But Can They Scale It?
The clock is ticking when it comes to generative artificial intelligence adoption. While businesses are bullish about it, implementations may tell a different story.
6 Min Read
Aisha Nuraini via Alamy Stock
There’s no denying GenAI’s popularity. Enterprises are adopting it in various ways, such as experimenting with it, upskilling employees, and adopting it by default since so many applications now include a GenAI feature.
The number of organizations able to get GenAI into production at an enterprise level is debatable. Both global professional services network KPMG and data science platform provider Dataiku recently conducted surveys about GenAI adoption. KPMG’s findings are bullish; Dataiku’s are sobering.
What KPMG Research Says
KPMG recently published its second AI & Digital Innovation Quarterly Pulse Survey, which reveals dramatic shifts just from the previous quarter. For example:
ROI metrics are rapidly evolving with most business leaders prioritizing revenue (52%) over productivity gains (40%).
The emphasis on hiring tech professionals has more than doubled from 26% to 60% quarter-over-quarter.
Simultaneously, organizations are heavily investing in upskilling their existing workforce, with training and capability-building initiatives jumping from 35% to 59%.
An overwhelming number (91%) of business and C-suite leaders identified technology, media, and telecommunication (TMT) as the sector leading in GenAI adoption.
“What really stood out to me is how many affirmative responses we got on this topic around are you doing things? Are you training people? Are you hiring differently? Do you plan to put this in your products? The numbers just have accelerated in every single category,” says Mark Gibson, national sector leader for TMT at KPMG.
The seismic shifts in the survey results reflect GenAI’s rapid adoption, and the fact that enterprises are scrambling to transform AI into a competitive advantage.
Gibson sees changes occurring in three different categories: adding value, workforce impact, and using GenAI responsibly.
Meanwhile, KPMG clients are asking for help with strategy, specifically around planning. They want to know how they should be thinking about AI in terms of how their companies will benefit and they want guidance on the inclusion of AI in the products they produce. They also need assistance with compliance and cybersecurity. Gibson anticipates the next wave of client inquiries will be about how AI, including GenAI, fits into their existing IT stacks.
“I used to focus all my time on the tech companies. Now, I'm getting asked to spend all my time on other sectors like energy and healthcare. They all want the tech person’s views of their business because they’re all becoming sort of tech companies,” says Gibson. “I think the tech companies are unique is, because they’re both trying to deploy [GenAI] inside their organizations, but they’re also the provider to all the others. So, I spend time on multiple axes, almost three dimensional, when I think about these conversations.”
Most management teams and employee bases in the TMT sector are ready to go, but they’re also giving themselves lower success scores.
“The velocity is high [in the TMT sector] because the acumen is there, and they’re critical. They’re also looking to prove they’re responsible for this security layer that is so important,” says Gibson. “And, board members often are leading from, how do we know this isn’t going to get us in trouble more than how do we know this is a way to really accelerate the value of the organization? So, I think tech companies are getting pulled in on the trust side, because they’re going to be asked, ‘How do you protect this?’”
To that point, there are a lot of startups focused on the security layer.
“I heard one person say, as much as 60% of their investment is going to be in this sector. And so, we need interest rates to come down and we need this money to be [used] to accelerate this forward, because right now the R&D is being done by those who can afford it: the big tech companies,” says Gibson. “They have all the money. So, we need the venture backed markets and emerging tech to get funded, so they can get out and drive a lot of this stuff as well. And I think that’s when it really takes off.”
What Dataiku Research Says
Dataiku’s research presents a more sobering view: Only 20% of senior IT leaders are using GenAI in production because they’re struggling to manage the complexities, such as scaling GenAI programs, achieving AI regulatory compliance, and reining in the lack of governance and usage control.
From an investment perspective, most IT leaders are planning to spend more than $500K over the next 12 months, but 88% have experienced infrastructure barriers to using large language models (LLMs) in the way they desire. According to 45%, data quality and usability remains the biggest data infrastructure challenge that IT leaders face.
“[E]very time there’s a new technology in this space, people want to skip over the hard part,” says Conor Jensen, global field CDO at Dataiku. “I think that there are some really cool applications that people are just not thinking about, because they still don’t invest enough in the dirty backend work of platforms, data cleaning, data cataloging, and stuff. It's the sewage work of data that nobody wants to do.”
Some of the other barriers Jensen sees are the short tenures of CTOs and the C-suite generally because by the time data owners get up to speed on what data they have and how to use it, they’re halfway or more than halfway through their tenure. Another challenge is data governance because there are a lot of nuances to consider, such as the fact that data governance and model governance are separate things.
“[M]anaging ML was hard, but at least it was somewhat deterministic. Obviously, with the generative AI models, now that’s completely gone. So, setting up systems for golden prompt and answer validation and verification and all of that [is hard because] there’s not even an industry standard for people to shoot for,” says Jensen. “What it means to actually govern the use of data and the use of these models is moving really, really fast, certainly much faster than most big companies are ever used to moving from a technology perspective.”
One Dataiku customer faced electricity supply challenges because they’re hosting their own machine learning models.
“So, this isn’t, 'we don't have enough servers, or we don't have enough data to do this stuff.' They’re running up against the availability of electricity in their country to run the scale of compute that they forecast coming as they continue to expand their stuff,” says Jensen, “So, while the current crop of ML technology is amazing, and has obviously done things that we didn’t really expect, I certainly didn’t expect from a compute efficiency perspective that there’s still a long way for us to go to get where we want to be. There are scalability challenges, [because] we’ve made it too easy to scale data. And that's causing all these downstream challenges. But the scalability of the inference right now is the number one hurdle that people are coming up against using this newest crop of technology.”
Bottom Line
Enterprises are scrambling to harness the power of GenAI, but they need help doing it. They need to know how the technology can benefit their companies and their customers, where it fits in their existing tech stack, how to plan for success and how to implement successfully.
It appears that some data fundamentals are getting in the way when it comes to scalability, however.
About the Author
You May Also Like
More Insights
