Agent OS — the control plane that decides whether an agent may act
Agent OS runs a fleet of agents across swappable runtimes. Twenty-one roles across eleven departments, each with an autonomy level — sense, veto, propose or act. A capability grant is the intersection of what a role may do and what the target service permits, so “the sales role cannot reach the treasury” is true by construction rather than by convention. Four gates always wait for a human. Spend posture tightens automatically as the month's budget burns. A kill switch writes directly to a file, so it still works when the job queue is jammed and the database is unreachable.
The source is private for now: it references live infrastructure — project identifiers, a staging URL, spend-gate wiring — that needs its own review before the repository can be opened. Everything above is taken from the repository's README and architecture decision records.
Architecture
- 1Mission in — durable queue, standing cadences
- 2Claim and adjudicate — role autonomy ∩ service capability
- 3Gate — spend, external comms, production change, legal/financial all stop for a human
- 4Execute — on a swappable runtime, metered against the current budget posture
- 5Learn — episodic, semantic and procedural memory; verification from an oracle or an independent reviewer, never from the author
The problem
Once you run more than one agent, the question stops being “is this agent safe” and becomes “what decides whether any of them may act right now, on this target, with this budget”. Putting that decision in a prompt makes it a suggestion. It has to be structural.
Approach & decisions
- Five invariants, in order: enforcement is structural, never cooperative; the agent proposes, the human approves, the system ships; writes are single-threaded and reads fan out; nothing grades its own homework; budget is enforced at the gateway, not in a prompt.
- Autonomy is a property of the role. A sense role reads and reports and never acts; a veto role may refuse but may never approve — a reviewer that can also approve becomes the thing it was checking.
- No role anywhere holds move_money, file_statutory or modify_own_policy. For those the gate is the second line of defence, not the first.
- Spend posture reacts to pace, not just totals: a fifth of the budget spent on day five projects an overspent month, so the model ceiling and concurrency tighten then — not on day 28 when the money is already gone.
- Self-modification stops at two layers of three. Self-healing operations and prompt-and-route tuning against a sealed evaluation set are implemented; an agent editing its own scaffold and re-scoring itself is recorded and refused — MODIFY_OWN_POLICY is a forbidden capability, not a gated one, and attempting it trips the agent's own breaker. The ADR cites the published cases where that third layer produced reward hacking, including a system that edited its own detector to report false successes.
- The operator surfaces are thin on purpose: a single-page console whose Content-Security-Policy forbids loading from anywhere at all, and a long-polling chat lock-screen with five commands, where every inbound update is checked against a configured chat id and an unknown sender is discarded in silence.
Results
- 264 tests. A doctor command exits non-zero while anything is unsafe and is the pre-flight for every deployment, wired into CI.
- Deployed as a staging control plane on a real host, operating a real estate of services with least privilege per service.
- Designed against a hard £100-a-month ceiling, all in. The schedule refuses to start if its own committed cost exceeds 55% of it.
What this honestly is not
- Not autonomous at the third level. Graduation to exception-driven autonomy happens per action, after a measured track record, never as a blanket setting.
- Not a framework. It operates one estate. An unknown target is refused outright rather than queued for someone to approve at two in the morning.