*Part 12 of An OS Where the AI Is Never a Principal.*
A security design that doesn't publish its limits is selling you something.
So this last post is the limits. Not the limits we ran out of time to fix — the ones we *can't* fix, because they are properties of the problem, not defects in the design. I ran three adversarial passes over Aegis looking for them. Nine survived. Each one is scoped around, never solved: there is a discipline that bounds it, and there is no patch that eliminates it. Telling those two things apart is the entire point of this post.
The previous post closed Act III by showing the same confinement move winning in a place that has nothing to do with AI — an untrusted smart contract that can't steal, only refund. This post closes the series by being honest about the ground the move never takes.
The nine
1. Free-text confidentiality cannot be soundly enforced. A model mixes its entire context into one latent state, and it can be steered to adversarially encode — steganography, paraphrase, an acrostic. There is no sound general declassifier for natural language. You cannot mechanically prove a free-text summary doesn't leak a secret it was shown. We live with it by going integrity-first, allowing confidentiality flows only for *structured* data with a sound structural redactor, and preferring prevention — never co-mingle — over detection.
2. You cannot revoke knowledge. Authority is revocable; information is not. Once the model has seen a secret, it's in context, in logs, in memory. Dropping a membrane stops future *actions* — it never un-tells a *fact*. That's a law of information, not a bug. We live with it through decay: make the only secrets the model sees ephemeral by construction, short-TTL and single-use, so a leaked copy is dead on arrival.
3. Reality-bound secrets leak eventually. Decay works for system-issued secrets — rotate a token to worthlessness. It does nothing for a secret whose value is bound to the world: a medical fact, an M&A target. Those don't expire on rotation. Given enough agents and enough time, they leak and they don't decay. We live with it by treating data minimization as a security primitive: don't let the model learn the reality-bound secret unless the task truly needs it.
4. Security bounds blast radius, not competence. Every guarantee in this series is about what an agent *can't do*. None of them makes it *correct*. A perfectly confined agent can still be wrong, useless, or manipulated within its authority. Ocaps bound the damage of a bad decision; they don't make the decision good. The model can be small and dumb *because* safety doesn't depend on its competence — but usefulness still does.
5. Both minimizations trade against usefulness. Least authority versus usability, least knowledge versus competence — these are fundamental tensions, not bugs. Withhold too much and the agent is secure and useless. Grant too much and you widen the blast radius. There is no setting that maximizes both. We live with it by budgeting authority and knowledge as resources: spend the least that clears the task's competence floor, found empirically.
6. The human-attention budget is finite and load-bearing. The human is the root of trust and the final arbiter of the hard calls — grants, declassification, labeling. That makes human attention a single, low-bandwidth, fatigue-prone resource that *every* gate draws on. Powerbox fatigue is real: click "allow" enough times and you've trained yourself to stop reading. You can minimize the draw; you cannot remove it without removing the human from the root — which removes the legitimacy. We live with it through a bounded human-interrupt budget — batching, durable grants, surfacing only outcome-changing decisions — over an unspoofable trusted path so each decision the human *does* make is real.
7. Something must be trusted. There is no zero-TCB system. The control plane that enforces every property is itself the universal point of total failure — one bug there and all attenuation collapses at once. Its build is a total-compromise surface; runtime confinement does nothing about a compromised confiner. Trust is *reducible* — a smaller, separately-verifiable core, the capability microkernel we keep pointing at — but never *eliminable*. We live with it by shrinking and attesting the trusted base, and by hardening the trusted components' own input-handling, since they are confused-deputy targets too.
8. Shared hardware leaks below the logical layer. On one machine, vats share caches, memory, and — most acutely — a single inference engine, with a shared KV-cache and batching. Microarchitectural side channels can move information across vats that are perfectly isolated *logically*. These are bounded, not closed, short of separate physical hardware per trust domain. We live with it by refusing shared inference state across trust domains, climbing the isolation ladder to microVMs, and documenting the residual instead of denying it.
9. Covert channels through the model's behavior remain. Even with labels and prevention, a model that has seen a secret can leak through *timing*, through *which tool it chooses*, through subtle structure in otherwise-clean output. Explicit-flow control doesn't touch these. We live with it by rate-limiting and output-coarsening to bound the bandwidth — and again by decay, because a leaked ephemeral secret is worthless however it escaped.
The shape of the residue
Now look at that list again and notice what isn't on it.
The authority half of the design — ocaps, vats, membranes, revocation, the whole apparatus that makes "the model is never a principal" true — has *zero entries*. It survived all three passes. There is no irreducible limit on the claim that a confined agent cannot escalate authority, because that claim is structural: you cannot command what you were not handed, and no amount of cleverness inside the model conjures a reference it never received.
Every single irreducible limit lives in one of the other three halves:
- Knowledge is irreversible. (1, 2, 3, 9.)
- Humans are finite and load-bearing, and the system can be useless or dumb. (4, 5, 6.)
- Something physical and something trusted always remain. (7, 8.)
That is the honest map, and it is the emotional payoff of the whole series. Aegis can make an untrusted agent provably unable to escalate authority. It can make exfiltration structurally hard and the exfiltrated thing worthless. It *cannot* make information reversible, humans infinite, hardware perfectly isolating, or the trusted base zero.
Here is the part that should make you trust the list rather than suspect it: those four things aren't on our roadmap because they aren't on *anyone's* menu. They are not problems Aegis failed to solve. They are problems no design solves, because they are the shape of reality. The honest thing a security system can do is push every hard problem out of the half it controls and into the halves nobody controls — and then say so out loud.
Back to the first sentence
Twelve posts ago, in the first post, I made a claim that reads as either obvious or heretical:
Stop trying to make the model trustworthy. Make the model irrelevant to security.
Everything since has been the working-out of that one move, all the way down to a kernel you can run. And the limits above are its vindication, not its refutation. The reason the authority half has no irreducible limits is *precisely* that we never asked the model to be trustworthy. We never put it in the loop that decides what's permitted. It proposes; the capability graph disposes. A thing you've made irrelevant to a property cannot break that property — not when it's brilliant, not when it's hijacked by the last web page it read.
You do not secure an AI agent by making the model trustworthy. You make the model irrelevant to security, and then you are honest about the limits that remain — which turn out to be the limits of information, of human attention, and of physics. Those were always going to be hard. The achievement is moving everything else out of their company.
I built the whole thing, it's open source, and I didn't ask you to take any of this on faith. Every claim in this series is backed by code you can run — two dozen runnable scenarios, a real local model whose every action is structurally constrained, an isolation ladder that ends at the formally-verified seL4 microkernel — and the core invariants were thrown sixty thousand random adversarial cases by a property-based fuzz harness that found zero counterexamples. The machinery is the same machinery from the Endo and Agoric tutorials. This series was what it's for.
That's the series. If you want the whole arc in one place — the idea, the build, the payoff — it's in the index.
The model is never a principal. Everything else followed from there.
End of Transmission.