Empire of AI: Dreams and Nightmares in Sam Altman's OpenAI

By Karen Hao

12 min read

Why does it matter? Because every idealistic decision OpenAI made had a shadow version designed to serve the same small group of people.

You probably think OpenAI's story is a cautionary tale — idealism that couldn't survive contact with capital, a safety board that voted to fire its CEO and then reversed course within five days, good intentions that curdled under competitive pressure. That framing is wrong, and Karen Hao spends this book proving it. The nonprofit charter, the alignment researchers, the staged reluctance to release dangerous models — none of it was idealism that failed. It was architecture. Every structure built to constrain power quietly served to concentrate it, and the people writing the safety memos and the people signing the Microsoft checks were running the same play. What Hao offers isn't a more nuanced version of the story you already have. It's a different story entirely — one where the mechanism is visible, the extraction is named, and the question isn't whether AI is good or bad but who, specifically, it is good for.

The Nonprofit Was Always a Power Move

In November 2023, as OpenAI's board teetered on the edge of destroying the company it had been created to protect, one of its directors offered a statement that deserves to be read twice. Helen Toner, a think-tank researcher and one of three independent board members, told the executives confronting her that if their decision to fire Sam Altman caused OpenAI to collapse, that outcome could still be fully consistent with the organization's mission. In another room, an employee wept over the prospect of losing millions in equity.

The line sounds like fanaticism. But Toner wasn't going rogue — she was reading the manual. OpenAI had been structured, deliberately, so that its governing board had sweeping authority to act in the interest of its mission rather than its shareholders. That mission was to ensure that artificial general intelligence benefits humanity. The phrase is so broad it can justify almost anything, including corporate self-immolation. What looks like a safeguard against Big Tech greed was also a safeguard against anyone — employees, investors, partners — ever having a legitimate claim on the organization's direction that could override the board. The mission language didn't constrain power. It concentrated it.

The original framing was pitch-perfect. In 2015, Elon Musk and Sam Altman gathered a group of researchers at the Rosewood Hotel on Sand Hill Road — the symbolic heart of venture capital — and proposed building a nonprofit AI lab that would be the anti-Google: open, collaborative, motivated purely by humanity's benefit rather than shareholder return. The logic had genuine appeal. Google had just absorbed DeepMind. Someone responsible needed to race them. A nonprofit, Altman argued, was the structure that could ensure the technology belonged to the world.

Within two years, the founders privately acknowledged that the openness framing was marketing that could be walked back as circumstances required. Within three years, Altman had created a for-profit arm to court outside investors. The architecture had been built all along to ensure that whoever held the mission statement held the keys.

Toner's boardroom declaration wasn't a betrayal of OpenAI's founding vision. It was the founding vision, expressed without the softening layer of good PR.

'Scaling Laws' Aren't Science — They're a Fundraising Argument

Scaling laws sound like physics. The name implies discovery — something found in nature, like the relationship between a star's mass and its luminosity. What Dario Amodei's team actually discovered in 2019 was closer to an accounting identity: if you feed a neural network more data, more compute, and more parameters in roughly the right proportions, its performance on next-word prediction improves along a smooth, predictable curve. That's useful engineering knowledge. But OpenAI turned it into something else — a mandate. A proof that the only path forward was more of everything, always, regardless of what 'forward' meant.

The sleight of hand required a specific setup. Ilya Sutskever had been preaching for years that intelligence would simply emerge from scaling simple neural networks — that the gap between a silicon node and a biological neuron didn't matter, that brains were just bigger computers. When Google's Transformer architecture appeared in 2017 and most researchers dismissed it as a niche tool, Sutskever evangelized it as the scalable architecture he'd been waiting for. A researcher who joined OpenAI around that time later described the push as 'a wack idea.' Sutskever wasn't deterred. He didn't argue his way to consensus. He kept repeating the same message — scale, scale, scale — with enough conviction that it became the lab's organizing doctrine.

Once scaling became doctrine, it created its own justification for cost. To feed GPT-3's ten thousand Nvidia chips, OpenAI abandoned the careful data curation that had defined GPT-2 and poured in Common Crawl — an indiscriminate scrape of the entire internet — along with what two people with knowledge of the training data described as torrented books pulled from Library Genesis, a site that hosts pirated books. The data got bigger and dirtier simultaneously. Cleaning the toxic outputs required hiring workers in Kenya at less than two dollars an hour.

None of this was forced on OpenAI by nature. The scaling laws didn't demand any particular dataset, any particular labor arrangement, any particular pace of deployment. Those were choices — made under the assumption that whoever reached scale first would define what AI became. The science didn't require the race. The race required the science to look inevitable.

The Workers Who Trained Your Chatbot Were Paid to Lose Their Minds

Mophat Okinyi was reading descriptions of parents raping their children when he fell in love. He had just met his neighbor Cynthia, was imagining a life together, had moved with her to a quieter neighborhood east of Nairobi where the buildings had more space between them. Every day at work he read five or six paragraphs of material categorized as child sexual abuse or bestiality, up to fifteen thousand pieces a month, sorted by severity according to instructions from a client he was never allowed to name. He earned between $1.46 and $3.74 an hour. He tried to compartmentalize. He couldn't. The content burrowed into him and followed him home, and within months he had withdrawn from everyone around him — stopped talking, stopped being intimate, stopped being the person Cynthia had married. When she finally left, taking her daughter and their belongings while he was out buying fish for dinner, she told him by text that he had changed, that she no longer understood him. He had no way to explain what he had been doing, or why, or for whom.

What Okinyi was doing, it turned out, was building the safety filter for ChatGPT. OpenAI needed a system to catch the worst outputs of its models before they reached users. To train that filter, it first needed human workers to read and classify hundreds of thousands of examples of exactly the content it wanted to suppress. Some of that material had been scraped from the darkest corners of the internet. Some had been generated by OpenAI's own software — researchers prompted a language model to produce detailed descriptions of specific grotesque scenarios, including instructions to write in the voice of a teenage girl posting in an online forum about cutting herself, because the filter needed training examples the internet hadn't supplied. OpenAI then contracted with an outsourcing firm called Sama to put that material in front of workers in Nairobi, under code names PBJ1 through PBJ4, at wages that averaged less than two dollars an hour. The workers signed nondisclosure agreements. They had no idea who the end client was.

This wasn't negligence — it was procurement strategy. Once OpenAI committed to training on data swamps rather than curated sources, the toxic outputs of its models had to be managed somehow, and the cheapest way to manage them was to find workers desperate enough to absorb the cost. Scale AI, the firm OpenAI later used for its RLHF work, had a playbook for exactly this: scout economies in crisis, offer high initial earnings to establish market dominance, then cut wages once workers had no better option. Kenya, with 67 percent youth unemployment, was one such target. When Kenyan workers eventually started using ChatGPT to speed up their tasks, Scale blacklisted them as cheats and exited the country entirely.

Okinyi eventually grasped the full shape of what had happened: his labor helped make ChatGPT safe enough to release, and ChatGPT then helped eliminate the writing income his brother Albert had relocated to Nairobi to pursue. He said he was proud of his contribution. He also asked whether his contribution had been worth what it cost him. Nobody at OpenAI had an answer, because OpenAI — which never paid Sama the full $230,000 it owed — had structured the transaction so the question could never reach them.

Safety Research Was the Competitive Moat, Not the Mission

OpenAI's safety apparatus worked the same way a pharmaceutical patent does: every genuine safety concern that surfaced inside the lab — and some were genuine — also happened to justify the exact competitive moves that Altman had already decided to make. Restricting research publications? Safety demanded it: the 'infohazard risk' of talking openly about AGI progress was too high. Slowing external access to model weights? Safety required it: bad actors, state sponsors, rogue developers. Pulling back from the open-source ethos of the founding years? Safety again, always safety. The concerns weren't fabricated. But they aligned too perfectly with the strategy of a company trying to build a monopoly in a winner-takes-all market.

The internal reality of that safety culture is best captured by what happened one night in 2019. A researcher working on reinforcement learning from human feedback — the technique designed to steer language models away from harmful outputs — flipped a minus sign to a plus sign before leaving a training run overnight. That single typo sent the process running in reverse. By morning, the model completed every prompt with sexually explicit language. The fix went into the codebase with a comment: 'Let's not make a utility minimizer.' It was funny, and it was also the entire state of safety research at the company that would go on to claim the moral authority to decide who should and shouldn't have access to powerful AI.

The effigy ceremony at Tenaya Lodge gets cited as evidence of how seriously OpenAI took existential risk. Ilya Sutskever in a bathrobe, dousing a carved wooden figure with lighter fluid in front of senior researchers, explaining it represented a deceptive aligned AGI. It is evidence of something, but not that. The researchers had not built an aligned AGI. They had not discovered it was lying. They had built a language model that produced impressive text, and one researcher had once fixed a typo. The ceremony dramatized the gravity of a situation whose gravity the participants had manufactured — and it happened at a luxury resort two miles from Yosemite.

The Planet Is the Infrastructure — And It Isn't on the Balance Sheet

When Microsoft trained GPT-4 in Iowa, its data centers pulled roughly 11.5 million gallons of water in a single month — about 6 percent of the local water district's entire supply — during a regional drought. That covers one month of one training run. Sam Altman was already planning further out: a proposed fifth-generation supercomputer requiring five thousand megawatts of continuous power, nearly matching the average draw of all of New York City. No one inside either company knew whether it was technically feasible. The energy had to come from somewhere, which meant the water did too, which meant the land did, which meant the minerals for the hardware did — and that supply chain runs straight through the Atacama Desert in northern Chile, the driest place on earth outside Antarctica, where a quarter of the world's copper and a third of its lithium are extracted from Indigenous territories hollowed out progressively since the Spanish arrived.

None of this is coincidence. Chile's extraction economy was the product of deliberate Cold War-era economic policy, not colonial accident. When the AI boom arrived, Chile was already the economy it needed to be. Communities in the Atacama watch their water tables fall, their flamingos disappear, their copper feed the cables that carry the power that runs the chips.

When a water activist group in Cerrillos, a working-class municipality outside Santiago, dug through a 347-page environmental filing and found that Google's proposed data center planned to draw 169 liters of potable water per second during a megadrought, they held a referendum and killed the project. Google sent engineers who spoke only English and offered to plant trees. The community understood the offer for what it was: the same transaction, softened with landscaping.

The Mission Statement Is the Con — It's Designed to Mean Whatever Power Needs It to Mean

The OpenAI mission statement is an engineering achievement. Not in AI — in power. It reads like a values document but functions like a blank check: 'to ensure AGI benefits all of humanity' contains enough interpretive slack to authorize almost any action while immunizing against almost any accountability. That wasn't an oversight in the drafting. It was the point.

Watch how it moved. In 2015, beneficial meant open-source and nonprofit, unconstrained by financial return. By 2020 it meant walling off the model and selling API access. By 2022 it meant racing to deploy ChatGPT as fast as possible. By 2024 it meant putting capable tools in people's hands 'for free (or at a great price).' By January 2025, Altman declared that AGI was now 'traditionally understood' to have been achieved, and the mission had therefore advanced to superintelligence — a target with no definition at all, which is to say a target that can never be reached and never be missed. Each redefinition came with its own moral urgency. Each made the previous version look naive. None required a vote.

The board crisis of November 2023 looks, from the outside, like a governance failure — safety idealists versus growth pragmatists, a messy dispute about leadership style. What actually happened is easier to see once you track the lie that sparked it. Altman had been trying to force Helen Toner off the board, using an academic paper she'd co-authored as his pretext. To build support, he told Ilya Sutskever that fellow board member Tasha McCauley agreed Toner had to go. Sutskever, suspicious by then of almost everything Altman said, called McCauley directly. She told him she'd said nothing of the kind — that the conversation Altman described had not occurred. A small lie, easily deniable, delivered verbally to exploit the fact that the two people involved almost never spoke. Except Sutskever had already started speaking to McCauley. The lie was caught.

That catch mattered not because it was unusual but because it was unusually confirmable. The WilmerHale investigation that followed Altman's reinstatement reportedly found numerous instances of him telling different people different things — a pattern that multiple senior executives, including Mira Murati and Sutskever, had spent years navigating before concluding, independently, that they didn't trust him to lead the company toward AGI. Altman returned anyway. The nonprofit's governing power was subsequently stripped and the organization restructured toward for-profit status, with a $6.6 billion funding round whose terms required the conversion within two years or investors could take their money back.

Every accountability mechanism had been neutralized. The mission statement made it look like continuity.

Even the People Who Left to Escape the Logic Rebuilt It

What if the people who left to expose the problem were already rebuilding it somewhere else?

When Dario and Daniela Amodei walked out of OpenAI in late 2020, they carried a specific grievance: Altman had been making commitments to Microsoft about model access that would make it nearly impossible to halt deployment if safety issues arose. In their telling, he managed dissent through the appearance of consultation while treating every decision as already made. They used the words

310 Hours of Consented Data Beat 680,000 Hours of Scraped Audio

In 2016, in the far north of New Zealand, Peter-Lucas Jones and Keoni Mahelona set out to build a speech-recognition model for te reo Māori — a language that colonial schooling had nearly erased, beating children who spoke it and reducing fluency from 90 percent of the Māori population to 12 percent. Their method was the precise inverse of how Silicon Valley builds AI. Before writing a single line of code, they asked their community whether this was even wanted. To collect training data, they ran an education campaign explaining what the data would be used for, then held a community competition. Within ten days, 2,500 people contributed 310 hours of high-quality audio — every participant consented. The resulting model achieved 86 percent accuracy using two GPUs. OpenAI scraped 680,000 hours of audio from the internet — without asking anyone — to train its equivalent tool, Whisper, its speech-recognition model.

The comparison does a specific kind of damage to a specific kind of argument. The standard defense of data extraction at scale is necessity: you need enormous corpora, you need massive compute, you need the resources only a well-funded lab can marshal. Te Hiku Media demolished that argument not by debating it but by doing something else entirely — and succeeding. The question was never whether AI could be built with consent and community governance. It always could be. The question is whether the people with the capital and the infrastructure had any incentive to find out.

That reframe — from 'is AI good?' to 'does this application concentrate or redistribute power?' — is the sharpest tool the book leaves in your hands. Not a policy proposal, not a boycott, but a test. Apply it to any system, any deployment, any deal: who ends up holding more? Te Hiku's answer was the community it served. OpenAI's answer, consistently, was OpenAI.

The Question You Should Be Carrying Out of Every AI Announcement

Here is the one question worth carrying out of this book: who controls it, who paid for it, and who couldn't say no to it. Ask those three questions about OpenAI in 2015 and you would have predicted the Kenyan content moderators, the Iowa water tables, the board coup, the mission statement that kept meaning whatever the moment required. The empire didn't emerge despite the founding ideals. It was constructed through them, one reinterpretation at a time. That leaves Peter-Lucas Jones and Keoni Mahelona — 310 hours of audio, every speaker a volunteer — as the closing image, not a counterargument but a proof of concept: it could have been otherwise. That fact doesn't redeem anything. It just closes off the last exit ramp — the one that says it couldn't have been otherwise. It could have been.