Bury the Right Corpse

Scott Aaronson has buried the idea of self-reference as key ingredient to intelligence. The argument: we built AIs that outperform most humans at most well-defined intellectual tasks, and at no point did anyone need to engineer in self-referentiality or strange loops. Hofstadter's thesis in Gödel, Escher, Bach goes to Westminster Abbey, alongside phlogiston and the aether, as one of history's important wrong ideas.

The burial is deserved. The headstone is mislabeled.

Ask what self-reference was ever accused of, technically. The charge sheet runs through the paradoxes, and the capital charge is Curry's: a self-referential sentence from which not merely a contradiction but everything, aka triviality, follows. Read the indictment closely, though, and Curry names a co-conspirator with a counter-intuitive name: contraction — the license to use an assumption multiple times having assumed it once. A small body of work (Rosenblatt, Weber, Roberts) keeps fully naive self-referential comprehension, diagonal constructions and all, and proves real (non-trivial) mathematics in it by policing contraction instead.

The two conspirators are, in fact, closer than accomplices. Category-theoretically, the diagonal map a ↦ (a,a) that powers Lawvere's fixed-point theorem — the abstract engine behind Cantor, Gödel, Tarski, and the halting problem alike — is contraction: to diagonalize is to use a thing twice. Roberts' recent work strips Lawvere's theorem down to its minimal substructural assumptions and finds the diagonal structure at the bottom. So the working option was never to delete the rule; delete it wholesale and you saw off the branch that fixed-point mathematics sits on. The surgical option, which is what the substructural theories actually implement, is to restrict contraction precisely on the Curry-shaped conditionals and leave everything else free.

Now run Aaronson's own observation through this. Self-reference, he notes, came free with universality: nobody built it into electronic computers or LLMs; it pops out as a byproduct of being able to talk about anything. And it cuts one step deeper than the post takes it: what comes free with universality also can hardly be removed. The diagonal lemma is not optional equipment; no design choice was ever available on that axis. Self-reference was never the interesting ingredient. Not because it is absent, but because it was not ours to decide. The structural rule is the only control surface available.

Why, then, does contraction sit quietly in every mainstream logic, every proof, every inference chain? Because it is what makes reasoning cheap. Prove once, reuse forever — that is contraction. The substructural systems that manage reuse are costly: metered reuse, re-derived lemmas (see Beall's detachment-free logics). I think it is time to budget these costs.

Looking at how agentic LLM deployments actually fail, contraction management becomes important. A prompt injection is a sentence about the context sitting inside the context — a fixed point of the wrong authority. What makes it lethal is not that it is self-referential. It is that the context grants it unlimited reuse: every subsequent step may attend to it, act on it, restate it as the model's own conclusion. A hallucination that hardens into a premise, a poisoned context that trivializes everything downstream. That is the anatomy of Curry's paradox rather than mere feedback: one bad premise, granted free reuse, spending the whole system. Generic data reuse does not produce that signature.

The mitigations that work already gesture substructurally, without saying so. Instruction hierarchies are stratification. Quoting untrusted text rather than obeying it is use/mention discipline. Taint-tracking is linearity by another name. Each was invented ad hoc, as a patch. Named properly, they are instances of one primitive: contraction management, that is deciding what in a context may be reused, how often, and with what provenance, before it is allowed to fire.

One thing should be said plainly, because it is the strongest reason to take the primitive seriously rather than the patches. There is no simple test that sorts the dangerous self-applicable forms from the harmless ones; the boundary is not decidable. So every concrete defense is, and will remain, a conservative over-approximation: it must over-restrict to be safe. That is not an engineering embarrassment to be fixed in the next release. It is the permanent condition of the problem, and it is exactly why reuse discipline has to be a first-class design primitive with an explicit budget, rather than a patch applied wherever last month's incident happened to land.

I regard contraction management as a load-bearing ingredient of agent security. The engineering question Aaronson's burial leaves open is therefore not how to keep the loops out of our systems. They were never out, and cannot be put out — dig for the ingredient and the grave comes up empty; structural inevitabilities cannot be interred. What his cemetery actually holds, phlogiston and aether and the rest, are ideas. The right corpse is one more of those: the presupposition, shared by Hofstadter's thesis and its burial alike, that self-reference was ever ours to decide. Put that in the ground, and the living question stands in the open — what a usage discipline over context looks like: reuse budgets, provenance-weighted attention, detachment gates on derived instructions, all deliberately over-tight. The loops are free. What they are allowed to spend is not.

A shorter version of this appears as a comment on Shtetl-Optimized.