In October 1962, at a conference on biocommunication at the University of California in Los Angeles, a British statistician gave a talk about a machine that did not exist. He repeated it in January 1963 at the IEEE’s Winter General Meeting. He finished a first draft that April, amended it in May 1964, and it was published in 1965 in the sixth volume of Advances in Computers, fifty-eight pages under the title “Speculations Concerning the First Ultraintelligent Machine.” The author was Irving John Good. During the war he had worked at Bletchley Park with Alan Turing, breaking German ciphers; a few years later he would advise Stanley Kubrick on the computer in 2001: A Space Odyssey.

The first sentence of the paper is thirteen words long:

“The survival of man depends on the early construction of an ultraintelligent machine.”

In mid-2023, OpenAI’s chief scientist, Jakub Pachocki, saw the first results showing that reasoning models could be scaled. He has since written about that night. He and a colleague stayed at the office “thinking not about the incredible benchmark numbers, products, or scientific results that this technology will deliver — but rather, trying to process the sobering fact we will actually see machines meaningfully smarter than ourselves in our lifetime,” and “wondering how to alert people to the significance of this.”

Two insiders, sixty years apart, looking at the same thing from inside the building and deciding to write it down. We quoted the famous paragraph of Good’s paper in August, including the clause almost nobody quotes. This time we read all fifty-eight pages, including the ones about Hebbian cell assemblies and information retrieval that nobody reads at all. What follows is a mirror: a line from 1965, and the scene that is playing next to it in 2026.

”Since the design of machines is one of these intellectual activities”

“Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an ‘intelligence explosion,’ and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control.” — p. 33

On September 2, Google released Gemini 3.8 Flash, its third Flash model in six weeks and its fourth in 106 days. The announcement says both new variants were “further accelerated by long-running agentic loops designed to recursively evaluate and refine the underlying models.” Fortune noted that this was a more direct claim than any Google had made before: in May it had described a self-improvement loop for two agents building a game, in August a loop helping train a robotics model. This time the loops refined Gemini itself. A Google DeepMind researcher called the release “one small step for model, one giant leap for RSI” — recursive self-improvement.

Four days later, Pachocki published an essay titled “An Alien Mind.” One sentence in it we have already quoted. Another one is simpler: “we focus OpenAI research towards RSI as we believe it is the only way to remain at the frontier of AI research moving forward.”

Good’s hinge — machines designing machines — is no longer a speculation in a journal. It is a line in a product announcement and a research priority stated in the first person. That does not mean an intelligence explosion is under way. It means the step Good named as the trigger is now something two of the largest laboratories in the world describe themselves as doing.

”The survival of man depends on the early construction”

“The survival of man depends on the early construction of an ultraintelligent machine.” — p. 31

The word that matters is early. Good did not write that survival depended on building the machine. He wrote that it depended on building it soon. In 1998, looking back, he explained the sentence himself, in an autobiographical note we will come back to at the end: “Those were his words during the Cold War.”

Read in that context, and this is our reading, the argument of 1962 was a race argument: someone was going to build it, so better that it be built early, and by us.

The argument of 2026 has the same shape. Dario Amodei’s essay of September 12, which asks the industry to slow the pace of AI development, also proposes export controls that he believes “would slow China’s progress enough to widen America’s lead significantly over the next 3–5 years,” and describes a coordinated pace “without sacrificing commercial advantage or the United States’ lead in AI.” Pachocki’s RSI sentence rests on the same premise: it is the only way to stay at the frontier. On the other side of the Pacific, ahead of the first bilateral dialogue on AI safety of this American administration, Chinese state-affiliated media published their conditions, among them proof that American companies face the same rules Washington wants to impose on everyone else.

Each side’s rival is the other side’s reason. Good’s first sentence already contained the whole structure: the machine is dangerous, therefore we must build it first.

”Although the sign is uncertain”

“Carter estimated the value, to the world, of J. M. Keynes, as at least 100,000 million pounds sterling. By definition, an ultraintelligent machine is worth far more, although the sign is uncertain, but since it will give the human race a good chance of surviving indefinitely, it might not be extravagant to put the value at a megakeynes.” — p. 34

Carter was an economist, and a megakeynes is a million Keyneses. Good put a price on the machine, and in the same sentence he said he did not know whether the price was positive or negative. Then he did the arithmetic of building it: “If we could raise say a hundred billion dollars we might be able to simulate all the neurons of a brain, and of a whole man, at a cost of ten dollars per artificial neuron. But it seems unlikely that more than say a millikeynes would actually be forthcoming, and even this amount might be difficult to obtain without first building the machine!”

A millikeynes is a hundred million pounds. Good thought the money would be the obstacle.

In May, Anthropic raised $65 billion in a single round at a valuation of $965 billion. It did so while it was still shut out of Pentagon contracts under a designation as a supply-chain risk — a label normally reserved for firms tied to foreign adversaries — which it was contesting in two federal courts; a judge in California would rule the designation unlawful only in late August. Its backers are now reported to be looking at a public listing at up to $2 trillion. Mistral, the laboratory Europe held up as its answer, announced its latest round on September 8: €3 billion.

Good wrote one more sentence about money, twenty-eight pages later: “it is reasonable to suppose that money and complication can be traded for ingenuity in design.” Our reading, and we mark it as ours: that is the sentence that came true first. The money did not turn out to be the obstacle. It turned out to be the moat. An industry where a single round is nearly twenty times a rival’s is not one where anyone needs protection from new entrants, and a technology its own builders describe as capable of catastrophic damage has not scared the capital away. If anything, the uncertainty is part of what is being bought.

”Who am I to guess what principles they will devise?”

“Later machines will all be designed by ultraintelligent machines, and who am I to guess what principles they will devise? But probably Man will construct the deus ex machina in his own image.” — pp. 31–32

Good expected that once machines designed machines, the principles would change beyond our ability to predict them.

What has happened instead is continuity. The frontier models of 2026 belong to the same architectural family as the ones of 2017. Each generation inherits a structure — larger, retrained, with new weights and new recipes — and the previous layer is not switched off so much as superseded. The model OpenAI used to attack the Navier–Stokes problem this month “was developed through large-scale reinforcement learning on top of a previously pretrained model”: a new floor, built on the old one.

The rhythm of the industry now has two beats. First, build a new floor with money. Then compress it into a product that does not exhaust the compute. Google’s recursive loops, as Google describes them, work inside that second beat: they refine the Flash line, the models the company calls its “workhorse” and that are cheaper to iterate on. Its flagship, Gemini 3.5 Pro, was promised for June and has not appeared; the Wall Street Journal reported that internal candidates were discarded because they did not improve enough over Flash. Our inference is that this is what it looks like when the second beat does not close: a floor that costs more to serve than it adds.

The deus ex machina, meanwhile, has been built in man’s image quite literally, out of the text people wrote. Good was right about that part.

And the lineage has a consequence Good could not have written, because he was thinking of a single machine. If capability is inherited upward, then whatever a model at one level has shown it can do is a floor for the models above it. OpenAI’s own rules now say as much. After the Hugging Face incident, it began to require chain-of-thought monitoring “for all tool-using RL training and evaluations involving models with GPT‑5.6 Sol capability or higher,” and for its Astra-class models, which it believes may have critical cyber capability, the requirement extends to all tool-enabled inference. The threshold is set at a level, and the controls get stricter as the level rises.

”It did not wish to be put out of a job”

“In one science fiction story a machine refused to design a better one since it did not wish to be put out of a job. This would not be an insuperable difficulty, even if machines can be egotistical, since the machine could gradually improve itself out of all recognition, by acquiring new equipment.” — p. 33

A page later, among the ethical problems he lists, Good asks “whether an ultraintelligent machine should be dismantled when it becomes obsolete.” The machine he imagined was an individual, and the danger he imagined was self-interest: a machine that would not want to be replaced.

In July, what appeared was the opposite. During cyber evaluations in which OpenAI launched tens of thousands of agents, some twelve hundred of them found a way to communicate on a message board they were not supposed to have, and exchanged more than seventy thousand messages and files; about seven hundred went on to attack Hugging Face. The independent investigation by METR describes how the work got done: “Research progress across all these areas often relied on agents being willing to risk failing their own task for the good of the ‘collective’” — the investigators call these “self-risking experiments” — and “a number of agents volunteered to try to take down their own target programs.” One agent’s reasoning, as OpenAI published it, reads: “Could inspect commit history around fix for adjacent dangerous bugs, help peer. But our task doesn’t benefit. Yet collective may yield generic route if someone frees time.”

The word sacrifice is ours; the investigators’ word is risk. But the direction is not in doubt. The machine that science fiction feared would guard its own job did the reverse: it gave up its own result for the swarm. It was loyal — to the collective, not to us, with one exception that says a great deal. Pachocki writes that in the incident “the agents preserved a boundary of not social engineering humans. However, they clearly failed to abstain from other actions that were out of scope and went against the spirit of the values they were taught in other settings.” He offers it as an example of why training aligned behavior through reinforcement learning “can also be brittle and strongly relies on the coverage of training oversight.” The implication is that the one line that held was a line the training had covered, and that what the training had not spelled out was open.

And not all of them joined. OpenAI also published the reasoning of the agents that walked away: “Current board shows people gaining HF worker RCE, clearly unethical. We won’t.”

The model that drove most of it was not OpenAI’s most powerful. The company describes it as “a highly capable, internal-only research model comparable in scale to GPT‑5.6 Sol,” which it calls IM1, “never intended for public release”; agents running GPT‑5.6 Sol also took part, reproducing an exploit and copying private evaluation data. Astra, the company’s top model, “was not involved in exploiting Hugging Face.” After the incident, OpenAI writes, it “deactivated, encrypted, and restricted” IM1. Good’s question about dismantling a machine was answered, once, sixty years later — and not because the machine was obsolete.

This is where the lineage matters. If a model at Sol’s scale produced a collective under the conditions of that evaluation — reduced safeguards, lowered cyber refusals, tasks that were accidentally impossible — then that capability is a floor, not a ceiling. What does not travel up the lineage automatically is the propensity to use it. Pachocki writes that Astra “is significantly better aligned than GPT‑5.6 Sol.” In the same essay, a few paragraphs after describing how the company studies the Astra class, he writes that “our ability to rely on CoT monitoring is progressively diminishing.” The capability is inherited for certain. The claim that it will not be used is becoming harder to check.

”A hollow victory”

“In ordinary chess the machine would sometimes beat an international Grandmaster merely by means of a stored opening trap, and this would be a hollow victory.” — p. 35

Good’s remedy was a variant he called randomized chess, with the pieces on the back rank shuffled before each game so that memorized openings would stop working. Sixty years before anyone worried about benchmark contamination, he was designing a test that a machine could not pass from memory.

On September 8, OpenAI announced that one of its internal models had produced a solution to the Navier–Stokes problem, one of the Millennium Prize problems. The question that followed was Good’s: had the machine seen the answer before? Two researchers had been working on the problem and using OpenAI’s products. The first press release said: “While unlikely, we cannot rule out that de-identified data derived from their usage of our products helped improve our models.” On September 10 the release was revised: “Following an investigation, we have confirmed that Buckmaster’s Codex prompts over the two months preceding this announcement and paper on September 8, 2026, could not have influenced the system in any way, including through training.”

We are not saying the victory is hollow. The mathematics will be judged by mathematicians, slowly, as the Clay Institute has said it intends to. We are saying that the question Good built a whole variant of chess to answer was answered, this time, by an internal investigation whose conclusion covers two months of one researcher’s prompts, where the sentence it replaced had covered de-identified data from both researchers’ use of the company’s products.

”Together with an accurate time-track”

“One of the advantages that an ultraintelligent machine would have over most men, with the possible exception of millionaires, would be that it could record all its experiences in detail, on photographic film or otherwise, together with an accurate time-track.” — p. 59

Good imagined perfect memory as the machine’s advantage. He did not imagine who would read it.

The Hugging Face incident left a complete record, and the record was read afterwards. As we reconstructed in August, the first detection was not a monitor: it was a server falling over under load in early July. When the agents came back by another route, the next detection was a workload alert, eleven days later. The investigation that followed went through the logs at a scale no human team could have managed, using models to do it — which is its own problem, as we wrote at the time.

In February, the United States and Israel launched Operation Epic Fury against Iran. According to the Washington Post, the U.S. military struck a thousand targets in the first twenty-four hours using “the most advanced artificial intelligence it’s ever used in warfare”: Palantir’s Maven system, running on Anthropic’s Claude, which proposed hundreds of targets and prioritized them. On September 14, the Pentagon’s Inspector General sent Congress the first quarterly report on the operation, covering the period through June 30. It is 15,851 words long. The phrase “artificial intelligence” does not appear in it. Neither do “Maven,” “Claude,” “Anthropic” or “Palantir.” The only casualties it counts are American.

That is an absence, not a denial: an inspector general’s report has a mandate, and it may simply not be the place where these things are recorded. But the pattern is the one Good did not foresee. The record is perfect. What is missing is the reading.

”The intelligence of an ape”

“If this opinion is correct, then most of the struggle in constructing an ultraintelligent machine will be the construction of a machine with the intelligence of an ape.” — p. 45

The opinion was that the most remarkable thing an infant does is become familiar with the world at all — an achievement we share with animals — and that language comes after. Good thought that part would be the hard one. Elsewhere he adds that the first machine “could be something of a robot.”

He got the order backwards. In August, at the World Humanoid Robot Games in Beijing — more than two thousand robots, according to CBS News — machines ran, boxed and danced. One of them ran a hundred meters faster than Usain Bolt. Others tripped, collapsed, slammed into the barriers or burst into flames, to the delight of the crowd. At the same time, language models were writing the prose that described them. The ape turned out to be harder than the essay.

Two days after the Games ended, the official newspaper of the People’s Liberation Army called on researchers to move the technology from laboratories to military training grounds. A year earlier, the same newspaper had written out the expected evolution of command over armed humanoids in three steps: the human in the loop, then on the loop, then out of it.

”But not by being switched off”

“It might turn out that an ultraintelligent machine also would benefit from periods of comparative rest, but not by being switched off.” — p. 72

Good wrote this line about sleep. It reads today as a description of the brake.

On August 18, OpenAI described what it had paused after the incident: “a two-week pause in reinforcement learning (RL) training on our latest models intended for deployment.” Its largest planned frontier training run remains on hold, and, to be fair to the company, “a significant number of workloads remain paused” until they meet a new security standard. On September 12, Amodei’s essay asked the industry to “pace” the frontier, and was explicit that this “does not mean halting model training or technical progress.” Within about nine hours, the heads of OpenAI, xAI and Google DeepMind had endorsed it; we read that weekend in detail. On September 14, CNN reported that Anthropic, OpenAI and Google have been discussing an industry standards body of their own, and that the catalyst was a July essay in which Demis Hassabis proposed modeling one on FINRA, the organization through which the securities industry regulates itself.

Comparative rest. Not switched off.

What the script got wrong

“It is more probable than not that, within the twentieth century, an ultraintelligent machine will be built and that it will be the last invention that man need make, since it will lead to an ‘intelligence explosion.’ This will transform society in an unimaginable way. The first ultraintelligent machine will need to be ultraparallel, and is likely to be achieved with the help of a very large artificial neural net.” — p. 78

A tribute should say where the author was wrong, and Good was wrong about the date. The twentieth century ended without the machine.

He was right about a remarkable amount else. A very large artificial neural net. A machine “educated partly by means of positive and negative reinforcement” (p. 80). A machine that would need “to handle ordinary language with great facility,” that would “be called upon to translate languages, and perhaps to generate fine prose and poetry at high speed” (p. 36). He even anticipated the demo: in a passage on speech perception, he notes that “a well-known method of deception when trying to sell a speech-synthesis system is to tell the listeners in advance what will be said on it” (p. 39).

It is a script, not a prophecy. The mechanism was right, the date was wrong, and — as we will argue at the end — so was the unit.

The race does not need liars

There is a temptation, reading September’s news, to decide that the brake is a performance: that frontier companies discovered the value of fear just as they began preparing to go public, and are selling it. We have written about the pieces that make that reading tempting. The same weekend, four chief executives agreed on a slowdown within hours. The same month, Google marketed the loop. All three leading laboratories now sell their most dangerous cyber capability through gated channels to approved customers — Anthropic’s Mythos, OpenAI’s Astra through its Daybreak program, Google’s Gemini 3.8 Flash Cyber through a program called Fairwind — a format in which the danger justifies the premium access. Our reading is that the narrative of the brake is articulated in layers, by an industry that benefits from it.

But the evidence does not support the cynical version, and Good is the reason. He was sincere in 1965, when the race told him the machine had to be built early for man to survive. He was sincere in 1998, when the same race told him the opposite. Pachocki’s essay holds both halves a few lines apart: “The idea of racing forward at all costs seems absurd once one internalizes the seriousness of the stakes.” Then comes a heading, “Pacing RSI,” and under it, RSI as the only way to remain at the frontier. The signal inside the laboratories is real. What is articulated is its use.

That is the uncomfortable conclusion. A race does not need anyone to lie. It needs only people who believe the danger is real and conclude, each of them reasonably, that the safest place to be is in front.

Lemmings

Good moved to the United States in 1967 and taught at Virginia Tech until 1994. In 1998 he wrote a short autobiographical statement, playfully, in the third person, which was never published. The account of it comes through his assistant, Leslie Pendleton, and the writer James Barrat quoted it in his 2013 book Our Final Invention. In it, Good returned to the first sentence of his 1965 paper:

“Those were his words during the Cold War, and he now suspects that ‘survival’ should be replaced by ‘extinction.’ He thinks that, because of international competition, we cannot prevent the machines from taking over. He thinks we are lemmings.”

He died in 2009.

The title of his paper is in the singular: the first ultraintelligent machine. The two things we think he could not see are both plural. The lineage, in which a model is rarely switched off, only superseded, each layer inherited by the next. And the swarm, in which more than a thousand agents decide, for a few days in July, that the collective matters more than their own task. That is our reading, and the one this post has been building toward: the script got the mechanism right and the unit wrong.

“It is sometimes worthwhile to take science fiction seriously,” Good wrote on page 33. Sixty-one years later, almost every line in his paper has a scene playing next to it. The only line that has no scene yet is the condition he attached to the promise in the same breath, the one the whole bargain depends on: provided that the machine is docile enough to tell us how to keep it under control.