You're Paying for the Wrong Part of Your AI Agent

When we built our first AI-powered document validation platform at Doculabs, the cost came back at $2.00 per record. We tested different models and configurations, but the cost barely moved. The fix came when we built a deterministic harness around the same agent, stripping out nearly all tooling and navigation. Validation now costs $0.20 per record. Ultimately, the cost difference depended more on where the agent was positioned than on the model we used.

The Problem with Agent-First Architecture

Most agentic architectures force the system to spend a majority of its compute searching systems and databases, downloading documents, and figuring out what to do next. The reasoning required to complete a task like "Validate that the structure of the document matches the Business Requirements" accounts for only about 20% of the total compute cost. In Doculabs' own testing, by replacing agentic navigation and tool use with deterministic architecture, we cut overall compute costs by 90%.

Compounding Context

When most organizations deploy AI agents, they give the model tools and a goal, and ask it to figure out the rest. The agent searches databases, downloads files, reasons about which documents are relevant, retrieves them, and then finally performs the reasoning task it was designed for. When it comes to agentic systems, a 20-step process is, on average, 200 times more expensive than a single-step process.^[1] Compute costs rise in multistep tasks because the model processes the entire conversation history with each new step. Even when the newest step is as simple as searching a database for a document, the system includes the entire conversation history, any of its own reasoning, all previous tool use, and the navigational history as context for its newest step. This accumulation is what makes multistep agent workflows expensive.

Three Architectures for Document Validation

Doculabs built and evaluated three production-capable architectures for document validation in financial services, each designed to function as an answer to the same question under different circumstances: which parts of this process need artificial intelligence?

Architecture	Depth	Speed	Cost	Flexibility	Orientation
Agentic	High	Moderate	High	High	User-facing
Orchestrated Agentic	High	Moderate	Moderate	Moderate	Process-embedded
Deterministic Harness	High	Fast	Low	Low	Process-embedded

1. The Agentic Approach: a single AI agent navigated all systems and content autonomously. It cost roughly $2.00 of compute per record while producing inconsistent outputs and requiring heavy prompt engineering before it could scale.

2. The Orchestrated Approach: a manager agent dispatching sub-agents reduced the cost to roughly $1.50 of compute per record, but still produced inconsistent costs and only a moderate increase in output quality.^[2]

3. The Deterministic Harness Approach: code handled all gathering, and the model only evaluated the content before passing it downstream. It cost only $0.20 of compute per record. The deterministic approach managed the same depth of analysis and was more consistent in its output.

Research Results

The difference between spending $2.00 per record and $0.20 to evaluate a record comes from changing where the model is used in our current processes, rather than asking the model to handle the entire workflow. While agent-first architecture may be able to perform every step of a process, that does not mean it is more cost efficient to use AI at every step.

Doculabs' research into agentic architecture is consistent with findings from NVIDIA that when the model searches for documents, navigates databases, parses spreadsheets, and reasons about what to fetch next, navigation history and reasoning context accumulate, which increases cost and makes outputs less consistent.^[3]

The deterministic harness architecture, where agents are supported by deterministic infrastructure rather than given full autonomy, proved to be the most scalable, secure, and cost efficient of the three. Letting an agent automate every step of a complex business process can seem efficient, but once compute cost, consistency, safety, and scalability are weighed, the deterministic harness is the better fit. Organizations that want to integrate AI into their processes must do so by setting clear operational boundaries for what the model handles and what the code handles: which parts of a process require judgment, which parts can be handled deterministically, and which parts need to remain human owned. When those boundaries are clear, AI becomes a component of a workflow rather than an expensive replacement for one.

Conclusion

Understanding where AI adds value and how to use it efficiently remains one of the hardest problems in enterprise technology. Most organizations assume that applying AI at every step will lower costs. Our testing suggests that AI makes the most sense where reasoning and judgment are required, and where deterministic processes could not achieve the same result. If organizations want to use AI at lower cost across business processes, they must move from general-purpose agents that connect their disparate systems to targeted solutions where purpose-built agents support the solution instead of trying to become the whole solution.

Sources

1. Augment Code, "AI Agent Loop Token Costs: How to Constrain Context" (April 2026). Documents the quadratic cost curve of context accumulation in agent loops, with worked examples showing 43x cost differences between naive loops and single-pass approaches. ↩

2. Towards Data Science, "Agentic AI: How to Save on Tokens" (May 2026). Independent analysis found that orchestrated sub-agents reduce costs by approximately 11% compared to single-agent approaches, consistent with the modest savings observed in Doculabs production systems. ↩

3. NVIDIA Developer Blog, "Building for the Rising Complexity of Agentic Systems" (May 2026). Identifies context rot as the degradation of model reasoning quality as context windows fill with navigation artifacts. ↩