The Big Threat to AI: Looming Disruptions

There’s plenty of worry about AI threatening mankind. But it’s AI that’s most under threat at the moment -- starting with problems in the semiconductor supply chain. That can be disastrous given the rapid upswing in AI adoption. AI requires “not just one chip and a few components, but rather hundreds of components to build the servers that can perform AI Large-Language Models and other training and inference,” says Scott Almassy, partner and semiconductor trust solutions leader at PwC US.

Even so, “there is nothing broken” according to Brandon Kulik who leads the Semiconductor Industry segment at Deloitte. Instead, the supply chain issues stem from “extremely high demand.” But that high demand is mostly funneled through a single source.

“Presently, supply availability is challenged since the overall sourcing largely runs through one supplier, Nvidia, and 100% of its manufacturing is located in Taiwan. Right now, there are very few alternatives or other solutions in the overall supply chain,” says Mark Dohnalek, president & CEO of Pivot International, a leading US-based global manufacturing, engineering, technology company.

There are attempts to address the situation, however. The CHIPS Act, for example, seeks to broaden semiconductor production, namely to the US.

Related:As Tech Layoffs Continue, Chip Foundry Plans Become Vital

“This would help lower costs. However, it will likely be 3 to 5 years before we see any impact from the CHIPS Act, as much of the funding for it has not yet been distributed, so US production hasn’t increased yet,” says Puneet Saxena, Corporate Vice President, Manufacturing Industry Strategy at Blue Yonder, a global supply chain operating system recently purchased by Panasonic.

And those aren’t the only threats to AI production. Several external factors, like geopolitical conflict, are also pressuring AI production. Other supply chains such as open-source software are added concerns as well. So, where does that leave all the companies trying desperately to put AI to work in their companies and products?

Semiconductor Supply Chain Woes

“GPU chips are in short supply relative to demand,” Saxena says. And that’s not likely to change anytime soon since adding production capacity takes years.

“While new fabs are being built and qualified, they will not be productive for a while. Until that time, squeezing every bit of efficiency out of existing supply chains is a priority for semiconductor companies,” Saxena explains.

The obvious question is what answers might AI produce to help solve this problem?

“There is a bit of irony that the semiconductor companies who are fueling a revolution in generative AI technologies are themselves underutilizing more practical AI capabilities, such as supply chain design and optimization as well as machine learning,” says Nari Viswanathan, Senior Director of Supply Chain Product Segment at Coupa.

Related:What Do US Restrictions of China Sales Mean for the AI Chip Industry?

By all accounts, the challenges are complex and there is no one solution to fix them all.

“There really isn’t one component that is not at risk in some way. We have seen gases used in manufacturing wafers become at risk, the export of rare earth metals used in manufacturing being restricted and we have seen a restriction on exports of chemical elements - all due to external circumstances,” says Almassy.

“We have heard anecdotes about one- or two-dollar parts holding up entire system implementations. So really with the surge in demand and the fact that the global supply chain is not fully caught up, all components could potentially be at risk in one way or another,” Almassy adds.

Open-Source Software Supply Chain Issues

As if semiconductor supply chain issues weren’t enough of a problem for AI production, other supply chains are piling on the challenges.

"AI is software and open-source code makes up 90% of most codebases, which means the open source software supply chain has just as much, if not more, impact on AI production than regulated hardware components,” says Feross Aboukhadijeh, founder and CEO of Socket.

Related:AI Chips Shortage a Growing Concern

The impact is potentially widespread given there are many open source AI models and tools on the market today and more are coming.

Examples include Hugging Face Transformers, Stable Diffusion, MindsDB, Fast.ai, OpenCV, GPT Engineer, Open Assistant, Fauxpilot, TensorFlow, PyTorch +Keras, Apache MXNet, tflearn, and many other models, tools, and ML libraries.

External Pressures Bearing Down on AI

External, global pressures are increasing both in number and intensity.

“The semiconductor industry has faced disruptions due to various factors, including the COVID-19 pandemic, geopolitical tensions, trade restrictions, natural disasters, and supply-demand imbalances. These disruptions have impacted the production and distribution of chips, affecting industries reliant on them, including AI,” says Shashank Agarwal, senior decision scientist at CVS Health.

While this general list is disconcerting, the specifics look even more worrisome.

“The wafer is the key -- the basic design unit. Only a handful of companies make these -- the biggest one is TSMC which makes NVIDIA’s GPUs. Everyone in the industry worries about China invading Taiwan leaving TSMC a smoking ruin. TSMC’s new Arizona plant is one step to onshore this capacity for the US, but it’ll be years before that plant is operational,” says Rhonda Dibachi, CEO of HeyScottie.com, a manufacturing-as-a-service company.

Labor, materials, and standard business issues also come into play.

“There are a number of threats to the supply chain, including concentration of semiconductor manufacturing in a few countries, shortage of semiconductor engineers, rising cost of materials, increasing complexity of chips, geopolitical tensions and regulatory and compliance challenges,” says Aarti Dhapte, Senior Research Analyst at Market Research Future (MRFR).

There are numerous efforts afoot to relieve these concerns and secure a prime slice of the AI market pie. For what corporation does not envy Nvidia right now?

“Many countries are trying to increase their piece of the global supply chain capacity and/or to onshore as much as possible through subsidies and other incentives. This has spurred significant investment and activity, but it remains to be seen whether these investments will address the supply chain problems in a timely or appropriate manner,” says Almassy.

Despite large investments, Almassy says PwC has already seen the ongoing talent shortage slow efforts to scale up these investments quickly. This all-too-common scenario, he says, would seem to indicate that any alternatives or solutions would be difficult to implement.

“Even if AI started to be used to revolutionize and optimize the global supply chain, physical investments would still be needed, so these constraints would still apply,” Almassy says.

The Forecasts

While supply chain status sounds all doom and gloom, there is light at the end of the tunnel.

Rand’s forecast for 2023 and beyond sees an end to supply chain issues in the short term.

“While we can’t precisely predict the future of the supply chain, a reasonable estimate could be around 5 to 7 years. This considers the time it usually takes for new products to be developed and made available to the public. The supply chain isn’t set in stone; it’s quite complex due to various factors that play a critical role when global production starts to ramp up,” says Jennifer Strawn, head of global solutions and sourcing for Rand Technology.

Other industry watchers are more specific in their forecasting but still see some form of near-term relief.

“At this time, most analysts expect manufacturing capacity for GPUs and HBM3 -- and a kind of next gen HBM3 called HBM3e -- to grow fast enough to meet demand,” says Kulik.

Further Kulik says that based on very recent announcements, Deloitte expects that:

1) CoWoS ((Chip on Wafer on Substrate) capacity at the company that currently does all the packaging for the most popular chips will more than double this year, and likely double again next year, and…

2) that technology will soon be qualified at other packaging companies (external companies, who specialize in assembly, test and packaging), which will significantly alleviate the current packaging bottlenecks.

“But all of this takes time, and almost everyone expects that demand for gen AI optimized GPUs will be greater than supply for Q3 2023 (100% certainty), Q4 2023 (almost certain), Q1 2024 (highly probable) and even Q2 2024 (probable, but less certain),” says Kulik.

Still others think AI will right its own ship -- in one way or another.

“The AI-focused semiconductor supply chain is remarkably flexible and has the potential to adapt along many dimensions, such as the types of data on which models are trained, the size of the models, and the efficiency of training and usage. A prime example of innovation in this space is the open-source project llama.cpp, which enables Large Language Models to run on CPU only, bypassing potential GPU supply bottlenecks,” says Nathan Schurr, CEO of AGI Technology.