Why Most Agentic Architectures Will Fail

Agentic artificial intelligence is expected to have a major impact because it can execute complex tasks autonomously. For now, the hype is outstripping successful implementations, and there are a lot of reasons for that.  

“In 2024, AI agents have become a marketing buzzword for many vendors. However, for user organizations, agents have been an area of early curiosity and experimentation, with actual implementations being far and few,” says Leslie Joseph, principal analyst at Forrester. “We expect this to change in 2025 as the technology and the ecosystem mature. However, our prediction offers a cautionary note.” 

Joseph says organizations attempting to build AI agents are failing for three main reasons: a poorly scoped vision for agentic workflows, a poor technical solution, and a lack of focus on change management. 

“A poorly scoped vision for agentic workflows results in either a too broad or narrow bounding box for agent functionality,” says Joseph. “Too narrow a scope may render the problem as solvable by a deterministic workflow, while too broad a problem might introduce too much variability. Agent builders should ask themselves how best to define the business problem they are trying to solve, and where an AI agent fits into this scope.” 

Related:What Could Less Regulation Mean for AI?

Second, it’s early days. Agents are still very early-stage applications, and the ecosystem, including agentic tooling, is less evolved than one might expect.  

“While many vendors message around the ease-of-use and drag-drop nature of their agent builder platforms, the fact is that there is still a lot of engineering needed under the hood to deliver a robust enterprise solution, which requires strong technical skills,” says Joseph. 

Finally, a lack of focus on change management isn’t helping. Organizations need to understand how the agentic workflow fits into or enhances existing processes and being proactive about managing change.   

“The invention of LLMs was like the discovery of the brick,” says Joseph. “With agents, we are now figuring out how to put these bricks together to construct homes and cities and skyscrapers. Every enterprise will need to identify what their desired level of autonomy is, and how to build towards that using AI agents.”    

Leslie Joseph, Forrester

He expects the short-term benefits to be process improvement and productivity, but over the longer term, enterprises should be ready for agents to create disruptions across the tech stack. For now, companies should embrace AI agents and agentic workflows, given its disruptive potential. 

Related:AI-Driven Quality Assurance: Why Everyone Gets It Wrong

“Start investing in experiments and allocating budgets towards proofs-of-concept. Ensure that your teams learn along the way rather than outsourcing everything to an ISV or tech vendor, because these learnings will be crucial down the road,” says Joseph. 

Multi-Agent Workflows Are Challenging 

When establishing a multi-agent workflow, there are three primary challenges businesses face, according to Murali Swaminathan, CTO at software company Freshworks. First, it’s incredibly difficult to make workflows predictable in a world that is unstructured and conversational. Second, even complex reasoning in workflows can be prescriptive and hard to achieve reliably. Third, continuous evaluation of these workflows is necessary to measure, and ultimately realize efficacy.  

“[E]nterprises must establish clear approaches on what workflows or problems they want the agentic systems to solve,” says Swaminathan. “Additionally, it’s critical that they develop a clear plan on how they will gauge success. This approach will ensure that expectations are measured, and that a strategy of ‘progress over perfection’ is employed.” 

Over the short term, enterprises will most likely achieve task-based goals related to the employee and agent. Over the long term, business benefits should follow, along with insights about what the business should and should not do. 

Related:6 AI-Related Security Trends to Watch in 2025

“[C]reate a clear game plan on how to implement, utilize, and measure the success of agentic architectures,” says Swaminathan. “Failing to plan is planning to fail."   

Insufficient Infrastructure and Data Governance 

When it comes to agentic architectures, infrastructure and data governance matter greatly. 

"Without the right infrastructure and data governance in place, agentic architectures struggle to handle the complexity, scale, and interoperability needed for successful implementation,” says Doug Gilbert, CIO and chief digital officer at experience-driven digital transformation partner Sutherland Global. “Companies should focus on building a strong digital core that can handle the high demands of AI, from data processing to seamless integration with hybrid or multi-cloud environments. This not only allows organizations to scale AI capabilities efficiently but also ensures the flexibility to adapt as systems evolve.” 

Equally important is a well-defined data strategy. Whether leveraging a hybrid, private, or multi-cloud approach, secure and accessible data is essential for building robust AI solutions, ensuring compliance and security across the board. 

Interconnectivity Matters 

Interacting with other systems designed for humans is much harder for agentic AI to do than it seems.  

“Making RPA [Robotic Process Automation] nearly 100% reliable took 12-plus years. And that’s carefully hard coded to interact with human operated systems across the web and Windows. So, we see these people suggesting that they can get an LLM to do the same and it turns out [to be] quite unreliable,” says Kevin Surace, chairman and CTO at autonomous testing platform Appvance. “People will be disappointed when the agent thinks it did everything right, but you later find that payment never went out.” 

Despite the fact humans don’t get everything right, people expect agentic AI outcomes to be 100% accurate. As an accuracy benchmark, Surace suggests setting the accuracy goal as high as RPA or well-trained humans. 

“Anyone can demo a simple action a few times,” says Surace. “But doing complex tasks with variability a thousand times without failure -- then you have a product people want.” 

Orchestration Can Be Tricky 

Orchestration involves end-to-end harmonization of outputs from multiple agents, delivering a unified and comprehensive resolution to the user’s query.  

“A key of the agentic AI architecture is its capability to organize agents logically by functional domains such as IT, HR, engineering, and more. This structured approach empowers enterprises to deploy specialized agents tailored to the unique requirements of each department,” says Abhi Maheshwari, CEO of agentic AI provider Aisera. “By categorizing agents based on their functional areas, organizations can optimize workflows, improve task precision, and ensure that each agent operates within its area of expertise for maximum effectiveness.” 

Otherwise, it may be tempting to over-rely on generic models when domain-specific expertise is necessary for handling complex tasks. 

“Enterprises should adopt a structured approach to agentic architectures by starting with logical domain separation to address specific departmental needs,” says Maheshwari. “Then there needs to be integration with existing systems. If not, there is not much value with agentic AI. After all, this technology is about automated processes and tasks.”   

Tom Taulli, author of Building Generative AI Agents: Using LangChain, LangGraph, and AutoGen, says that agents struggle to handle novel situations or inputs outside their training data. 

Tom Taulli, Author

“Failures can also arise from misaligned goals or insufficient oversight,” says Taulli. “Overly autonomous systems without proper guardrails might make decisions that conflict with user intentions, ethical standards or operational objectives.” 

Ultimately, enterprises need highly qualified data scientists because agentic AI is complicated, and the field is constantly evolving.   

In the short term, Taulli expects agentic AI to replace RPA since both automate tedious and repetitive processes. However, RPA is highly constrained, which means if a process changes materially, then the bot can break.  

“[Agentic AI] should be able to adapt and evolve, without a lot of human intervention and programming or scripting. This makes the automations more maintainable and scalable. “In the long-term, I think AI agents could start replacing a large part of what some employees do. This is more about the world when AGI starts to emerge. That is, a super intelligent system will be able to act like a human and have its own agency.”  

Governance Is Key 

Data quality and quantity are crucial for training agentic AI models, though biases in the data can lead to biased and unfair outcomes while ethical considerations and regulatory challenges surrounding AI development and deployment can hinder progress and lead to unintended consequences. 

“It's essential to establish robust governance frameworks to ensure ethical AI development and deployment,” says Matthew Hawkins, chief technology officer at healthcare AI solution provider CaryHealth. “Additionally, collaboration with domain experts is crucial to align AI solutions with real-world needs.” 

Monitoring Is Non-Optional 

The only way to tell if an AI model is working properly is to monitor it continuously. Otherwise, companies run the risk of using models that have drifted, for example. 

“When we look at AI agents as a system, the lack of human supervision can create devastating failure cascades throughout the entire network of agent,” says Daniel Clydesdale-Cotter, CIO at technology services provider EchoStor. “Domain-specific LLMs won't be good enough to replicate business process workflows without individual agents being thoroughly tested and optimized to remove hallucination behavior. The black box nature of LLMs adds another layer of complexity, as audit and compliance of operations within each agent can be very difficult to provide.” 

Organizations must focus on training and culture change to promote responsible use of generative AI for baseline processes.  

“It’s crucial to experiment, use, and test with existing workflows before integrating agents, always maintaining human oversight,” says Clydesdale-Cotter. “Organizations must also monitor their AI environment closely, aware of outputs and behavior within the system. Success depends on aligning data, goals and objectives with the usage patterns of the environment.” 

Success starts with human oversight and a defined MLOps plan, he says. Organizations should partner with companies that are building agents specifically for their domain requirements. However, they must also pay thorough attention to workflows to determine application and software integration viability.  

“Enterprises should approach AI with a best-fit mindset, understanding that not all processes must be AI-augmented or automated,” says Clydesdale-Cotter. “Being use-case specific helps avoid scope creep and maintains focus on the features you’re trying to extract. We’ll see continued process improvement through human oversight of macro interactions between AI agents and unsupervised optimizations of micro-processes within AI agents themselves.” 

Many Organizations Just Aren’t Ready 

AI has become a strategic priority in many organizations, but business leaders aren’t sure where to apply AI to solve day-to-day business problems and implement use cases at enterprise scale. 

“Under the hood, the challenge is that, even though business objectives, activities and metrics are deeply interwoven, the software systems used by disparate teams are not, and this creates problems,” says Babak Hodjat, CTO of AI at multinational information technology services and consulting company Cognizant Technology Solutions. “This is a big reason why we’ve seen most AI use cases to date limited to prediction-based outcomes or single LLM chat-based solutions." 

Organizing overall technology and AI strategies around the core tenet of multi-agent systems and decision-making will best enable enterprises to succeed, he says. 

“LLMs are very good at specialized tasks, but embracing multi-agent architectures is what will truly reshape industries, as agents gain the ability to communicate with each other,” says Hodjat “The future will be about companies having agents in their devices and applications that can address needs and interact with other agents. These agents will work across entire businesses to assist humans in every role, from HR and finance to marketing and sales.”