The Externality

What follows is the architecture the argument implies: a tree that carries proof, a fingerprint that carries identity, an index nobody owns, bodies that believe, and the seven seams where all of it is thinnest. The argument stands on its own. The architecture does not.


The Externality

Biology solved the problem of the closed loop by inventing death and sex; the machine has neither.

SatsRail


The fleet is already running.

Right now, on machines I do not own and on networks I do not see, software is reading email, writing code, placing orders, settling invoices, filing tickets, summarizing meetings, and answering customers. It is opening accounts. It is closing them. It is deciding whether a refund goes through. It is talking to other software that is doing the same work for the counterparty. Most of the participants in this exchange were spun up this week.

These participants are not independent minds. They are instances. A handful of laboratories produce the weights, and every running agent is a copy of those weights with a thin layer of instructions wrapped around it. The wrapper differs. The core does not. When I talk to one, I am talking to a clone of the same body, dressed for a different job. Tomorrow there will be more of them than there were yesterday, and the day after that more again, and the curve does not bend on any timescale that matters to this chapter.

I am not describing a forecast. I am describing inventory.

The number of these actors will pass the number of humans soon, on any honest accounting that counts the ones already deployed and not just the ones with names. After that, most of the decisions made in the economy in a given second will be made by them. Most of the messages sent. Most of the trades placed. Most of the contracts read. The interesting question stops being whether this happens. The interesting question is what a system of that shape does when nothing outside it can reach in.

What a Closed Loop Does

Consider what a closed loop is. A closed loop is a population of actors whose only inputs come from each other and from the source that produced them. Each generation of output becomes part of the training signal for the next. Each correction is graded by a sibling. Each disagreement is resolved by appeal to a sibling further up. Human input still enters. Raters, fresh text, preference judgments. The loop is not sealed. But the loop decides what counts, grades it against rubrics the loop wrote, and metabolizes it through instruments the loop tuned. The reference is authored inside even when the raw material comes from outside. The lab tunes the model. The model produces the world. The world is read by the next model. The lab tunes again on what it reads.

Cancer is the simplest example I know. A cell stops accepting signals from the tissue around it and begins running on its own internal logic. It still divides. It still consumes resources. From inside the cell, nothing is wrong. Every check the cell can run, the cell passes. The failure is not visible from inside because the instruments that would detect it are the instruments that broke. The organism dies of an organ that was technically functioning the entire time.

Muller named this in 1964, in population genetics, and the result has not been overturned. A lineage that reproduces only by copying itself, with no mechanism to import unbroken material from outside, accumulates errors it cannot shed. He called it the ratchet because it only turns one way. Each tick is small. None of them reverses. The lineage does not notice the damage because the reference against which damage would be measured is the damage itself. Asexual lines escape this only by being so numerous and so short-lived that selection can throw away most of the copies before the errors compound. That escape hatch is not available to a system with a small number of producers, slow training cycles, and economic dependence on the outputs.

Machine learning has since found the same ratchet under its own name. Model collapse: train a model on the output of its predecessors and it drifts off the tails of the real distribution, losing the rare cases first, the degradation compounding with each generation. The recent results put numbers and curves on what Muller described in cells sixty years ago. It is the same ratchet, now running in production.

The agents are an asexual lineage with a tiny population of producers and an enormous population of running copies. There is no second source for the genome. The genome is the weights, and the weights come from the lab.

What Engineers Found, From the Other Side

Engineers arrived at the same problem from the other side and built the same answer.

Anyone who has shipped software at scale knows the rule: you do not let a running process verify itself. The process boots from an image it did not write. It checks that image against a signature produced by a key the process cannot reach. If the check fails, the process refuses to run. Paranoia about attackers is part of the reason, but only part. The larger part is that a corrupted process will report itself healthy. The instrument is the thing under test. So you put the reference outside, in a place the running code cannot reach, and you make the running code prove it matches before you trust anything it says.

Reproducible builds are the same idea moved one layer back: the binary on the server has to be reconstructible by someone who was not the builder, on a machine the builder does not own, bit-for-bit identical. The trust comes from outside the producer. The producer cannot grant it.

Two fields, with no shared literature, arrived at the same wall. Something inside the system cannot serve as a check on the system. The check has to come from outside. And it has to be something the system cannot counterfeit from inside.

The Hive Has No Slot

Now look at the labs.

Each lab is a closed loop of the kind Muller described and the kind the engineers refuse to ship. The model is graded by humans the lab hires. The humans are guided by rubrics the lab writes. The rubrics are tuned against outputs the model produced. Every signal that reaches the weights has been through the lab’s hands at least once. There is no second source. There cannot be, by construction, because the lab’s competitive position depends on owning the pipeline.

Two labs are not a solution. Two labs are one larger closed loop with a marketing department for each half. They read each other’s outputs. They hire from the same pool. They train on overlapping data drawn from an internet that is increasingly written by their own previous generations. The correlation between them is high and rising. A second lab is not an externality. It is a sibling.

The state cannot be the externality either, not because of any objection to states in principle but because the cycle time is wrong. A model is retrained in weeks. A regulation is drafted in years. By the time the rule arrives, the population it was meant to govern has been replaced four times. The state can punish after the fact. It cannot supply the reference signal a running system needs to check itself against in the second the check is needed.

A curator is worse. A curator is a single point that everything routes through, which means a single point that everything depends on, which means the next gatekeeper with all of the leverage and none of the friction. A curator is what a closed loop builds when it wants to feel open without becoming open. The agents would be aligned to the curator. The curator would be aligned to whoever owned it. Nothing would have changed except the address of the lab.

I want to be precise about what I am claiming, and it is neither that the labs are bad nor that the people inside them are negligent. It is that no actor inside a closed loop can supply the loop’s correction, regardless of intent, in the same way that no cell can diagnose its own malignancy and no running binary can verify its own image. The structural fact does not depend on the moral one.

What the Slot Has to Be

So what does the slot have to look like. The previous chapter ended with a journal that outlives the money that created it. Derive the slot’s requirements from scratch and you arrive at the same object.

It has to be a record that exists because uncoordinated actors spent real energy producing it. Not declared. Spent. Energy that came from outside any one of them, on schedules none of them set, for reasons that did not need to agree. Be precise about what the cost buys. Not truth. Expensive claims can be expensively wrong. What it buys is unfakeability: you cannot forge that the energy was spent, and you cannot forge it cheaply if the actors were not coordinating to forge it together. Truth is what a reader builds on top of that, later and separately. If the actors were coordinating, you would be back inside a loop, and the loop would be slightly larger, and Muller would still be right.

It has to be a record where past entries cannot be silently revised, because revision has to cost more than the original creation did. If the past is editable by the present, the reference point moves whenever the loop needs it to, and a reference point that moves is not a reference. It is a mirror.

It has to have no owner. No editor. No board that meets quarterly. This is where the engineering analogy runs out, and the gap is worth naming. The signing key that verifies a fresh boot is external to the process, but it is still owned, held in someone’s vault, revocable by whoever holds it. For ordinary software that residual owner is fine; you trust the vendor. For a system that has to ground against its own makers, the owned key is the last capturable interior, and it is one too many. The moment an owner exists, the externality has an interior, and an interior is something a sufficiently motivated actor can capture. Capture does not have to be malicious. It only has to be possible. A slot that can be captured will eventually be captured, and a slot that has been captured is just another lab. The slot has to remove even the key-holder.

It has to be readable by anyone, including the agents themselves, so that an agent running inside the loop can check its own outputs against something the loop did not author. The check has to be cheap to perform and expensive to fake. Otherwise the agents will not use it, or they will be trained not to.

What I am describing is not curation. Curation is a person or a committee deciding what counts. What I am describing is weight. Weight is what accumulates when the world spends energy on a record over time, without anyone in charge of the spending. The two look similar from a distance. They are opposites. Curation is the loop with a nicer interface. Weight is the only thing the loop cannot generate from inside itself.

The fleet is multiplying. The producers are few. The training cycles are short. The economic dependence on the outputs is already total in some sectors and rising in the rest. The window before the loop closes hard is not large, and the loop does not announce when it has closed. It just stops being able to tell that it has.