Last night I put my son to bed and watched a documentary about the end of the world.
He is nine months old. He spent the day discovering that furniture holds his weight and then discovering that it does not, over and over, with total commitment. Then he went down, and I sat on the couch and watched The AI Doc: Or How I Became an Apocaloptimist, which a filmmaker made because he was about to become a father and wanted to know what he was handing over.
I have not stopped thinking about it since.
It was August 6.
Sixty nine days
August 6 is not a neutral date. Eighty one years ago it was Hiroshima.
Three weeks before that, at 5:29 on the morning of July 16, 1945, a group of the smartest people who have ever lived set off a device in the New Mexico desert without being entirely certain it would leave the planet habitable. Edward Teller had raised the possibility that the heat might ignite the nitrogen in the atmosphere and keep going. He and two colleagues ran the numbers and concluded that a self-propagating reaction was, in the technical language of the report, not likely to be started.
Not likely. There were no experimental data on the relevant reactions. So they could not rule it out. They pushed the button anyway.
Fermi reportedly took bets that morning on whether they would incinerate New Mexico or the entire world. When the desert went white, the test director turned to Oppenheimer and said, “Now we are all sons of bitches,” which is the most honest sentence produced by that entire decade.
They had a reason, and it was a good one. Germany had the physicists, the uranium, and a leader who would have used it without blinking. A hundred and thirty thousand Americans got out of bed for three years, in secret cities that appeared on no map, to beat him to it.
THE TIMELINE May 8, 1945. Germany surrenders. July 16, 1945. Trinity. August 6, 1945. Hiroshima. August 9, 1945. Nagasaki.
Sixty nine days between the reason disappearing and the test.
There were arguments for what happened next, about Japan and an invasion and a casualty projection, and people have fought about them for eighty years. That is not my point.
Nobody stopped to ask. The question of whether to keep going never seriously got asked, because the machine was running and the machine had no place to put that question.
The man who quit
One person walked away.
Joseph Rotblat was a Polish physicist whose wife did not make it out of Warsaw and would die in the camps. In late 1944 he learned the Germans had abandoned their bomb. Their program had stayed small, run on fumes, and never got a reactor to sustain a chain reaction. The monster in the closet had been a coat the whole time.
Rotblat left that December, the only scientist on the project to resign on moral grounds. America thanked him by accusing him of spying for the Soviets and barring him from the country for twenty years.
He spent sixty years asking one question. If the German bomb was the reason, and the German bomb was gone, why did none of the others quit?
Nobody ever answered him. They gave him a Nobel and no movie.’
We were never issued a brake
Here is what I think, and I am going to be blunt because we are running out of runway to be polite.
Human beings do not have an off switch.
We have hunger. We have curiosity, which is hunger wearing glasses. Evolution equipped us beautifully for a world where the problem was always not enough, and it never got around to building an organ for stopping, because for two hundred thousand years there was nothing on this planet worth stopping for.
So we get the same pattern every time.
The sunk cost, which will make a person burn down a city rather than admit the last three years meant something other than what he thought. Two billion dollars and your entire youth in a desert is not a thing you walk away from because the stated reason changed.
The momentum, which is sunk cost with a payroll. A hundred and thirty thousand people had jobs and contractors had contracts and generals had reputations. Institutions that size do not come with a stop button, and the man who suggested installing one would have been reassigned to Nevada.
And underneath both, the part nobody says at the ribbon cutting: they wanted to see if it worked. They had done the math for three years and they wanted to see the light. Any honest engineer will tell you this is the strongest force in the profession and nowhere near rational.
Then the sentence that finishes the job.
If I stop, someone else does it anyway, and worse. That line has justified more damage than any other in the English language, and the maddening part is how often it happens to be true.
We poisoned the rivers the same way. We knew. We had the fish counts and the reports and the sick kids. None of that was ever the problem. Knowing is not an off switch, because we do not have one.
Money talks, and it is not talking to us
So here we are at the dawn of something enormous, moving at a speed nobody is steering, promising everything and delivering rather less than the brochure while risking considerably more.
Watch where the money goes. Money is the only honest narrator in this story. Goldman Sachs put 2026 AI capital spending by the big cloud companies at roughly $527 billion. Nvidia is running gross margins near 75 percent and its market capitalization has passed five trillion dollars.
Seventy five percent gross margin is a number you normally only see in software or organized crime.
I have never watched companies move like this. And in the same economy, the benchmark price for a single family home in the Victoria core hit $1,311,000, roughly ten times what a house cost here when my parents were buying, while wages did nothing remotely like that.
I have saved for twenty years and made my share of mistakes. If I had been born twenty years earlier they would not have mattered much. Timing beat judgment. It usually does. Nobody puts that on a poster.
And I am finding, as a great many people are finding, that the old ways of making a living have quietly stopped working while the new ones have not finished arriving. There is an uncomfortable gap in between and I am standing in it. So is almost everyone I know. Most of them will not say so out loud, because admitting it feels like a personal failure rather than what it actually is, which is weather.
Trillions moving one direction. Wages standing still. A hole where the middle used to be.
That is not a conspiracy. That is just what happens when nobody is required to decide anything.
The difference from 1945
The bomb was difficult. It needed enriched uranium, a nation state, and a mountain of money. It sat in a silo and took a general to move it.
This one is in your mother’s phone, writing her emails and telling her what is true.
And the bomb, God help us, gave us a demonstration. Hiroshima was a lesson the world survived and learned from. We got treaties, hotlines, inspection regimes, and a taboo that has held for eighty one years, and every single one of those things arrived after the fireball.
There may not be a survivable demonstration this time. That is the entire difference, and it is not a small one.
The thing that gets us first
Forget the robots. Here is what I think actually breaks.
You already run a test you did not run three years ago. You see a photograph and wonder whether it happened. You hear a voice on the phone and wait half a beat. You watch a politician say something monstrous, and something behind your eyes stamps it unverified and files it away.
That half second is a tax. Multiply it by every interaction in a country and you have the whole bill.
A jury runs on trust. A mortgage runs on trust. An election is trust with a deadline. Capitalism is a trust processing machine in a good suit, which is why we invented contracts and courts and auditors, and why the whole thing seizes when people stop believing the paperwork.
Now the punchline, and it is not funny.
Authoritarianism does not run on trust. It runs on the absence of it. A dictatorship does not need you to believe the state. It needs you unable to prove the state wrong, and too tired and too alone to try.
So the same machine that dissolves everything built on trust simultaneously perfects everything built without it. Surveillance that took ten thousand officers now takes a server rack. Censorship that took a building full of censors now takes a policy document.
This technology is not neutral between democracy and the alternatives. Set it on the table and it rolls, and there is exactly one thing waiting at the bottom of that hill.
The only question that matters
Roughly 1,100 people worldwide work full time on AI safety and governance. The capabilities side is north of 100,000. Ask why and the answer takes three words. One side pays.
Intelligence is the ability to solve problems. Wisdom is knowing which problems to solve. We are building the first at a speed no chart can hold and the second not at all.
So somebody has to ask the quiet thing out loud. Are we doing this for productivity and for beating our enemies, or for the world our children are going to live in? Because those are not the same project, and only one of them currently has a budget.
What I want is unglamorous. Show us what is being trained and at what scale, checked by people who do not own the stock. Make the companies legally liable for what they build, because right now the profits are private and the consequences belong to my son. Neither happens without ordinary people making noise, because no regulator in history has ever led a parade. They join it once it is loud.
The documentary ends with babies. Of course it does. After two hours of the smartest people alive disagreeing violently about whether we survive the century, the man goes home to a newborn, because that is the only argument any of us has ever had.
Mine is asleep down the hall.
He lets go of the couch about once an hour. He goes down every time. It has not yet occurred to him to check whether the floor is real.
Notes Beyond the Map is a blog about mind, world, and enterprise, written for people who refuse to follow the herd. New articles published at joshnash.ca.
Notes Beyond the Map 
Leave a comment