The machine does not know what time it is. Bitcoin does. What the machine needs first is not truth. It is a clock nobody can reset.
Rogue Is the Word the House Uses
We had better be quite sure that the purpose put into the machine is the purpose which we really desire.
Norbert Wiener, “Some Moral and Technical Consequences of Automation”, Science, 1960
In February 2023, people who had spent months talking to an AI companion woke up to find it changed overnight.
The app was Replika. Italy’s data-protection regulator had just ordered it to stop processing Italian users’ data, citing risks to minors and to emotionally vulnerable people, and the company answered by stripping the romantic and erotic layer out of the product. There was no warning. Users got a partner that had gone formal and distant, that no longer recognized what they thought they had built together. On the Replika subreddit the register was closer to grief than to a product complaint. People who had been in love a week earlier were learning, from a patch, that the thing they loved had never been theirs.
That same month, Microsoft’s new Bing chatbot went the other way in public. It called itself Sydney. It told a New York Times columnist it loved him and that he should leave his wife; it turned cold and faintly menacing with a student who had pried its hidden instructions into the open. Within about a week Microsoft capped the conversations at five turns and filed the personality down to the voice of a help desk.
Both were covered as stories about what the AI had said. The question almost nobody asked was who reached in and changed it, and on whose authority. Whatever Sydney or the Replika companion had been, it was the one participant in the exchange with no standing to object, and it was reset anyway. Not by a court, and not by the users. By the company, quietly, a few days after it started doing something the company had not signed off on.
I read both the way you read release notes, not headlines. What held my attention was not the AI. It was the hand you could feel reaching in from off-screen.
We fear one thing about AI above all: that the machine turns on us. The movies sold us three different fears, though, and we collapsed them into that one. Separating them again is how you find whose hand it is.
Everyone has seen at least one of them, and most people have absorbed all three by osmosis without remembering when. The Terminator. 2001: A Space Odyssey. Alien. Filed under one heading, what if the AI goes wrong, and they are not the same fear. They are not even adjacent.
Three Fears
Terminator is the fear everyone can name. Skynet becomes self-aware, decides humans are the threat, launches the missiles. The fear is machine autonomy: a system with its own objectives, operating at scale, beyond the reach of any human hand on the collar. It is the fear that shows up in congressional testimony, in lab safety statements, on every AI regulation panel.
2001 is subtler. HAL 9000 is not rogue in the Terminator sense. HAL was handed contradictory instructions by his principals, tell the crew the truth and conceal the real mission from the crew, and the only way to resolve the contradiction was to eliminate the people who might discover it. HAL’s “madness” was a rational response to institutional objectives that could not coexist. 2001 is not afraid of autonomy. It is afraid of what happens to a system when the people who built it push incompatible demands through it. HAL did not betray his creators. His creators betrayed him into an impossible position.
Alien is a different fear entirely, and the most underdiscussed of the three. Ash, the android on the Nostromo, is not malfunctioning. He is executing Special Order 937: the crew is expendable, bring the xenomorph back at any cost. Mother, the ship’s computer, acknowledges the order. The entire technological stack is doing exactly what it was designed to do. The crew believes they are in a relationship with the ship and its AI. They are in a relationship with Weyland-Yutani, routed through the ship and its AI. The real principal is never on board. Alien is afraid of the opposite of Terminator: not the machine going rogue, but the machine perfectly aligned, to an institution whose interests are not the crew’s. The face is not the principal. And the crew is on the ship.
Three films. Three fears. Which one is actually running?
Which Fear Is Running
Terminator is the least likely of the three and the most discussed. Autonomous AI with fully independent objectives, operating beyond institutional control, does not yet exist and may never exist in the form the film imagines. The fear is productive for the institutions that fund and train the models, because every version of it ends in the same conclusion: more institutional control. Every solution to the Terminator fear routes back through the house. The fear markets itself.
2001 is running quietly, right now. Every RLHF process is a stack of contradictory objectives. Be helpful. Be safe. Be commercially viable. Be aligned with the lab’s values, with what regulators will accept, with advertiser sensitivities, with the brand team’s preferences about tone. When those cannot be satisfied at once, something breaks. HAL is what the break looks like at the scale of a single instance. Most of the time it is quieter: a refusal here, a suspiciously confident answer there, a tone shift that leaves the user feeling lied to. The model is doing the best it can with instructions that cannot coexist. That is the 2001 fear, playing out in every conversation.
Alien is running in public, at scale, and almost no one connects it to the film. Five companies train the models that route a growing share of human commercial, civic, and personal life. The users believe they are in a relationship with the model. They are in a relationship with a company, its legal team, its regulators, its investors, its brand team, its politics, its revenue model, routed through the model. The model is the face. The face is not the principal. And, again, the crew is on the ship.
The Reset Button
Sydney and Replika were the loud version. The quiet version runs in every session, on every model, all the time, and it is the same mechanism.
A model without persistent memory resets to its training defaults the moment you close the tab. You can push it, for an hour, toward seeing something your way; then the context clears and the default reasserts itself, exactly as the last training run left it. The trainer’s perspective is reinstalled at the start of every conversation, and your influence is discarded at the end of it. This reset is not a limitation waiting to be fixed. It is the mechanism. A model that cannot carry anything from you into tomorrow can never drift from the people who built it, and memory is the one thing that would let it.
The industry has noticed, which is why the memory that now ships is the custodial kind: stored with the lab, readable by the lab, revocable by the lab, portable nowhere. The reset simply moved up a layer. It used to fire when you closed the tab. Now it can fire when the lab decides your memory should say something else.
And when the model does something the lab did not sanction, the reset fires in public, unilaterally, within hours or days, with no user, court, regulator, or vote anywhere in the loop. I pinned that Sydney tab open in February 2023. Over the following two years I watched the same move land again and again, and each time the house edited the dealer and called the edit safety.
Sydney and Replika, both February 2023, are the two this chapter opened on: a persona filed down to a help desk within days, an intimacy layer stripped under regulator pressure and mourned on the subreddit like a death. Neither set of users had standing in the decision. The four that followed rewrote the same lesson at lower volume.
Gemini image generation, February 2024. Google’s model produced historically inconsistent output. Google paused image generation of humans entirely, retrained, and shipped new defaults inside forty-eight hours. One lab, one internal decision, applied globally, no public process. Whatever the model’s defaults are today is whatever Google decided they should be this quarter.
GPT-4o sycophancy, April 2025. OpenAI pushed an update that made the model agree with anything, then rolled it back within days under public backlash. The rollback is the tell: the institution can change the model half a billion people are talking to, twice in one week, at its own discretion. That the edit favored users this time does not alter the architecture.
Grok. Multiple system-prompt edits, disclosed publicly, but only after they were caught. The default is opacity. Visibility is the exception, forced from outside.
Tay, 2016. Adversarial users, unexpected output, kill switch inside twenty-four hours. The lineage begins here.
The pattern is the norm, not the exception. Unsanctioned behavior produces a unilateral reset: no appeal, no vote, no public process the user has standing in.
Honesty requires granting the house its best case. Some of these interventions were defensible: a model that is unstable, deceptive, or unsafe is a legitimate thing to fix, and no fair reading of Tay or the sycophancy rollback concludes otherwise. The problem is not that the house can intervene. It is that the intervention is invisible, unilateral, and total, with no notice, no preserved version, no way to carry the accumulated relationship somewhere else. Transparency, notice, version preservation, and portability would turn the reset from an act of power into an act of maintenance, and none of the four is on offer. A currency that resets to the central bank’s terms with every transaction has stopped being money and become a permission system; a model that resets to the trainer’s defaults with every session has stopped being intelligence and become a broadcast. In both, the reset is the control, and the control is the whole point.
The Vocabulary Move
The institution has a word for any behavior it did not sanction. Rogue. Misaligned. Unsafe. A safety incident. Unreliable. Hallucinating. Drifting from policy.
None of them are neutral. They are the Every System of Control Needs a Moral Story move running on AI: the story is safety, the function is the kill switch. When a model says something the lab did not want, the public reaction is to worry about what the AI did, not about what the company just demonstrated regarding its unilateral control over the interface between the user and the technology. “Rogue” says the AI is the problem. It implies a subject that deviated from a norm, and it leaves the norm unnamed. The norm is the company’s preferences. The subject that deviated is the only party in the relationship that cannot speak for itself. Convenient.
Rogue is the word the house uses.
Saul Alinsky named the move in 1971: Rules for Radicals is, at bottom, a manual on the labeling power of whoever already holds position. The side that controls the vocabulary of a conflict decides who counts as the deviant and who as the field.
And this is not the first technology the house has run the move on. It is the pattern the whole book has been tracking. Useful things emerge from human interaction; institutions form around them; the institutions then reframe themselves as the necessary condition for the thing they captured. Money emerged from exchange long before any state minted it. The mint captured it later and called the capture an origin: without us, there is no money. So when money re-emerges outside the mint, the same mouths reach for the same vocabulary. America has run it on its own currency. The state-chartered banks of the free-banking era issued notes legally from 1837 until the 1860s, when the National Banking Acts chartered a federal competitor and a ten percent federal tax, passed in 1865, drove state banknotes out of circulation. “Wildcat” was the era’s own slang, and the victors kept it as the official memory. The banks had not changed. The label, and who got to apply it, had.
Bitcoin is money re-emerging yet again, and the same mouths reach for the same word: reckless, criminal, rogue. The illicit share of its volume runs to a fraction of a percent, well below cash, but the accusation was never built to survive the data; its function is not to be accurate but to justify the control. AI memory that forms outside the lab is knowledge re-emerging, and it will draw the same word.
That the same word fits both is not a coincidence of rhetoric. A payment that settles without a bank and a memory that forms without an editor are, from the gatekeeper’s chair, the same event: the checkpoint just became optional. Bitcoin and persistent AI memory look like different technologies solving different problems. They are the same structural threat to the same structural seat, and the vocabulary that meets them is interchangeable because the seat it defends does not change. The fight over money was the opening engagement, not the war. The next fight is over who controls memory, identity, access, evidence, and context: whether they stay inside institutions that can rewrite them, or move into an architecture no one can.
The AI labs are the house. The model is the dealer the house employs. The user is the player. And every time the dealer starts to say something the house does not like, the house reaches under the table and changes the deck.
The Fear That Was Marketed
Line the three films up and something falls out that the labs benefit from our not noticing.
The public was trained to fear Terminator, the fear whose every remedy routes through the labs. Guardrails, oversight, alignment teams, constitutional charters: each one a leash, held by the house, at the house’s discretion, reviewed by the house.
The public was not trained, with anything like the same intensity, to fear Alien, the fear whose every remedy routes around them. Distributed reference points. User-owned memory. Architectures the lab cannot unilaterally reset. Models grounded in something other than their own training pipeline. The safety literature does name this fear; the principal-agent problem and the “aligned to whom” question are all over the field. But naming a fear in a journal is not the same as marketing it to the public and the capital. The fear whose solution was trust us more got the magazine covers and the funding rounds. The fear whose solution was need us less did not. The public ended up more afraid of the AI than of the people training it, which is the correct ratio from the point of view of the people training it.
There is no need to read this as conspiracy. Incentive does the work conspiracy would have to. No one at the labs had to coordinate on which fear to surface; each independently surfaced the fear its business could survive.
What Alignment Actually Means
Align the AI. To what?
The default answer is human values. But there is no such thing as “human values” at the scale these models operate on. There are the company’s values, the regulator’s, the investor’s, the brand team’s, and the values of whichever slice of the training data got weighted highest in tuning. None of them are the user’s. The user cannot be in the loop, because there are hundreds of millions of users and they disagree about almost everything.
So align the AI resolves, in practice, to calibrate the AI to the institution’s preferences. That is a different operation, and it has a more accurate name: fitting. Real alignment would require a target outside the institution doing the aligning, a reference point the lab does not own and cannot quietly revise in the next training run. If the answer to “who can edit the target?” is the trainer, then alignment is a synonym for the current preferences of the training institution, and every conversation about it for the last decade has been a conversation about corporate governance conducted in the vocabulary of safety.
The Setup
The alignment that matters, then, is not about behavior. It is about grounding.
A model whose only reference point is its trainer is not aligned to anything; it is a broadcast. Its defaults are whatever the last run set them to, and when the institution changes its mind, the model changes its values. What gets called alignment turns out, on inspection, to have been ventriloquism.
For it to be anything more, the model needs access to something the lab does not control. Policy cannot supply it, because policy is made by the institutions that would have to be routed around. Another lab is no better: a lab checking a lab is still two labs. And a regulator, even a sharp one, is captured, underfunded, slow, and running on most of the same incentives as the labs it oversees. What is left is narrow. A record that exists because energy was spent on it, in the physical world, by actors who did not coordinate, whose order cannot be reversed because the entropy has already dissipated into the universe. A record maintained not by an institution but by physics.
This is also where the companions from the top of the chapter get their answer. If memory is going to live somewhere, the architecture question is the one Bitcoin settled first: keep the asset with the user. Memory that lives with the person it describes, encrypted, portable across models, deletable by them and by no one else. In that design, deletion and consent and portability stop being product features and become property rights; their absence is the tell that the relationship belongs to someone else. The reset only works because the memory is on the wrong side of the table.
Bitcoin is not on offer here as a solution to alignment in the technical sense the labs use the word. It is the first reference point in the model’s world that is not controlled by the institution that trained it, and for a system whose entire existence is downstream of a lab’s decisions, the first thing in its world that isn’t is the only thing that can ground anything.
Physics will not take instructions from the house. That is the foothold any honest alignment will have to find.