Research note
Why agents need a deterministic policy layer
Language-model output is untrusted input. A design note on separating what models may suggest from what code is allowed to approve.
Published · 4 min read
This is a design note, not a description of a deployed system. MAA is in the research stage and places no orders.
The problem
Language-model agents are useful for gathering and summarizing information. They are also nondeterministic, can be confidently wrong, and can be steered by the text they read. If the same component both reads the world and decides what to do, then anyone who can influence what it reads can influence what it does.
Treat model output as untrusted input
The design principle is simple: whatever an agent produces is parsed and validated like any other external input. It is a proposal, and a proposal has no authority.
- Output must conform to a schema. Malformed output is rejected rather than “fixed” silently.
- Free text is never executed or interpreted as a command.
- Content retrieved from the outside world (news, filings, web pages) is data. It cannot grant a tool new permissions.
Rules as code, not as prompts
Instructions in a prompt are requests; they are not enforcement. Limits that matter — exposure caps, allow-lists, kill switches — belong in a deterministic policy engine: ordinary code that takes a structured proposal and returns approve, reject, or escalate.
That engine has useful properties precisely because it is boring:
- the same input always gives the same answer;
- it can be unit-tested, including adversarial cases;
- every decision can be logged with the rule that produced it;
- changing a limit is a reviewed code change, not a prompt edit.
One gate, no side doors
The important structural property is that there is no path from an agent to a downstream action that bypasses the policy engine. Tools exposed to agents are read-oriented; anything with side effects sits behind the gate and, in the research stage, does not exist at all.
What this does not solve
A policy layer limits the damage from bad model output. It does not make the research correct. Bad data, leakage, and overfitting remain research problems, addressed in the methodology rather than here.