Could AI Cause a Global Financial Meltdown?

Innovation in artificial intelligence is unfolding rapidly, with the potential to revolutionize the financial services industry as we know it.

With the breakneck speed of development continuing apace amid a period of economic volatility -- most recently underscored by the collapse of banks in the United States and the rescue of major financial institutions abroad -- there are growing concerns over the role AI could play in further destabilization.

In May, Securities and Exchange Commission Chair Gary Gensler cautioned AI platforms could be central components of future “fragility” in the financial system and called for scrutiny of the use of Generative AI by financial institutions, lest it pose a “systemic risk” in the future.

“The risks and challenges associated with these emerging technologies are undoubtedly growing, posing concerns among regulators for their impact on existing financial systems,” explains Dennis Gada, executive vice president, global head of banking and financial services, at Infosys. “However, there are also several benefits to keep in mind.”

For example, financial institutions will gain efficiency and competitiveness, insights for better decisions, and improve client experience that will help them stay ahead in the game if they learn how to leverage AI.

“AI is quickly becoming an important technology in day-to-day work across the financial services sector, leading to higher productivity and faster delivery,” he adds.

AI and the Risk of Unknown Unknowns

Steve Sanders, chief information security officer for CSI, says the biggest challenge and resulting risk are both a result of the lack of insight into the decision-making of AI systems.

“Even when given well-defined parameters, AI systems can still generate unexpected responses,” he explains.

While one might think the financial system is based on facts and hard data, the latest economic challenges, which include the Federal Reserve’s efforts to prevent a credit crunch and fight high inflation, are one illustration of the complexity involved.

“The inputs and outputs of our financial system include an incredible number of situational scenarios and previously unencountered circumstances, which results in a very difficult training scenario for AI systems,” he says.

On January 26, 2023, NIST released the AI Risk Management Framework v1.0, and subsequently launched the Trustworthy and Responsible AI Resource Center on March 30, 2023, to support this framework.

“Experts agree that even if this voluntary framework were implemented, the opaque nature of closed systems will remain a challenge,” Sanders says.

He adds it is almost certain that AI algorithms will continue to become more complex as they further emulate human thinking.

“Even without the opaqueness, this creates a very difficult scenario for risk management,” he says. “AI systems will also continue to become more opaque unless transparency is forced through new regulations or laws.”

AI Finance Use Poses Challenges for Regulators

While AI applications for financial services raise challenges for regulators due to the unknown risks, Gada notes the same can be said for almost every industry applying this disruptive technology.

“Given the risks of using AI in financial services are more technical than those of other industries, there is heightened scrutiny among governing bodies, due to the sensitive nature of data, privacy and room for fraud,” he explains.

To circumvent these risks, regulators can implement AI auditing processes to provide standards, practical codes and guardrails for safe AI use in financial services.

He notes there are both upsides and downsides to regulating AI, and while regulation provides a framework for ethical practices, information security and consumer rights, it also poses a threat to the progression and innovation of this groundbreaking technology.

“Balancing regulation and innovation will be the best path forward to ensure safety, responsibility and value in deploying AI,” Gada says.

Sanders adds systems should be designed with a governance-first mindset, otherwise gaining control over the risks is going to be difficult.

“However, this is unlikely to be implemented without regulation or legislation, both of which introduce political and economic challenges,” he says. “Companies do not want to be late-to-market on AI technology, and governance is seen by many as a speed-to-market hindrance.”

He points out countries and geopolitical economic groups struggle to impose restrictions fearing long-term economic damage. “Companies will resist sharing too much about the inner workings of their systems, reducing competitive advantage,” Sanders says.

From his perspective, a nimble, well-represented task force representing banking and finance may be able to implement a mandatory framework and control structure around AI use in banking and finance.

Accounting for Bias in AI Systems

Meanwhile, ethical concerns around AI deployment remain unsolved, as regulators work through how AI systems can be held accountable for bias.

“Given the future of AI is still uncertain, and the technology is inevitably going to evolve in the coming months, there is room for financial leaders to influence regulation and guide regulators in making decisions on its future,” Gada says.

Sanders explains while many tout the ability of AI to reduce bias in decision-making, the design and training of these systems can both create intentional and unintentional bias.

“The intentional bias is easy to explain and a real risk as people often do not realize their own biases which could make their way into the decision-making system,” he says. “The unintentional biases are much more complicated to deal with.”

One example of this could result from a less banked people group, which as a result is less represented.

“The training data for this people group may not prepare the system to deal with the idiosyncrasies of that people group,” Sanders notes.

What to Read Next:

Confronting Financial Fraud in Payments with the Help of AI

AI, Data, and Crypto in Play at Fintech Innovation Lab Demo Day

IBM’s $4.6B Apptio Purchase Places Bigger Stake in FinOps

Payments and Productivity Tools Dominate Techstars NYC Demo Day