Section 8 – Limitations of Agentic AI and Why we still need humans

Agentic AI is powerful, but its effectiveness is shaped by four core constraints.

Reliability

Agentic AI systems are based on non-deterministic models. What that means is that outputs will always vary, even when you give them the same inputs.  And, while these systems are capable of sophisticated reasoning, they’re not always consistent. Variability in responses and the potential for incorrect conclusions will introduce risk, particularly in enterprise environments. 

Your agentic AI agent will work across multi-step workflows, meaning that even small errors here and there can add up and compound into bigger issues. 

To address that problem, it’s necessary to create:

• Validation layers to confirm outputs
• Structured workflows to guide execution
• Guardrails to constrain behaviors
• Normalized data to reduce ambiguity in responses

Note that reliability won’t come from the model on its own. Instead, you have to build a strong system that enforces consistency and correctness over speed or efficiency. 

Context

Another thing to be aware of is that agentic AI systems rely on the data they’re trained on. Data must be both relevant and accurate. If it’s not, the AI’s understanding of the situation isn’t going to be accurate and should be seen as incomplete. 

The problem is that in many organizations, procurement data is fragmented across multiple systems, which can lead to gaps. What the AI can find out about suppliers, financial data, contracts, or other data may be siloed, and that’s a problem. 

Fix the issue by:

• Unifying data across procurement systems
• Ensuring deep integrations to give the AI access to real-time visibility
• Focusing on building out consistent data models for improved clarity

Setting up a strong foundation for context makes AI better at decision-making and reduces limitations. 

Execution

Agentic AI is designed to take action on your behalf, but it doesn’t do that without the right infrastructure in place. Large Language Models, or LLMs, can’t interact with systems on their own. Instead, they need tools like APIs, workflow engines, and others to do the tasks they’re able to do. And, you likely already know that many AI implementations stop at the recommendation level — they do not execute, and they leave that work for human counterparts. 

To help agentic AI reach its full potential, you need a few things:

• A unified platform foundation that gives AI consistent access to workflows, data, and permissions
• Orchestration mechanisms that will manage task execution across processes
• Execution layers that will connect AI to your intended workflows

Once the groundwork has been laid, you’ll be able to transform your AI from a simple analytical tool to one that can actively manage workloads and tasks. 

Governance

Finally, you need to look at governance. As agentic systems gain the ability to act on their own, governance is a critical component to keeping it on track. 

Agents working within enterprise environments will have access to sensitive data, compliance obligations, financial transactions, contracts, and many other items. Maintaining proper control is essential because without that, there are risks like data exposure or regulatory violations that could reveal themselves. 

Good governance ensures that agents always operate safely and within the boundaries that you’ve defined for them. 

To establish a solid governance framework, you need to:

• Set up role-based permissions to grant, remove, and control access to data
• Create audit trails that track every action taken
• Manage monitoring systems to provide oversight 
• Ensure compliance by creating good policy enforcement techniques. 

Remember, governance is not optional. To have trust in an agentic AI system, it is vital to have a solid governance framework. 

That’s why human involvement is still so essential. Humans have the skills and knowledge to provide oversight. They can validate decisions and manage exceptions when necessary. They can also ensure that agentic AI systems don’t operate out of bounds and do remain focused on positive business objectives.