The university where the term artificial intelligence was coined is ringing in the field’s 70th birthday in fitting fashion: with an academic-integrity scandal. Last month, Dartmouth College’s president, Sian Leah Beilock, published an essay in The Atlantic headlined: “Why I Want More AI at Dartmouth.” Now Beilock is launching a review of whether her top deputy used too much AI.

On September 21, the student newspaper The Dartmouth reported that Provost Santiago Schnell appeared to have relied heavily on AI to write all nine of the articles he’s published this year—including a Washington Post op-ed about students outsourcing their writing to AI. Schnell’s response to the accusation didn’t help his cause. According to the student paper, he sent an email statement acknowledging that he uses AI chatbots as “assistive tools” in his writing but downplayed their role in his process. Even Schnell’s statement, the paper found, registered as “100% AI-written” on Pangram, an AI-detection service that is imperfect but boasts a very low error rate. A Dartmouth spokesperson told me that neither Beilock nor Schnell would comment for this story.

[Read: The pangram backlash unfolding on college campuses]

Schnell is the latest example of an ironic phenomenon: People whose job involves writing about the appropriate uses of AI—or even setting the rules on AI use for others—keep getting caught using AI to do it. There was the Australian university professor whose op-ed about the importance of students doing their own writing turned out to be at least partly AI-generated. There was the nonprofit executive whose book about the future of truth in the AI age turned out to include quotes fabricated by AI. There was the AI beat reporter who reportedly lost his job after he accidentally copied quotes invented by AI into one of his stories. And there was the Stanford misinformation expert who got upbraided by a judge when his declaration supporting an anti-AI-deepfake law turned out to contain ChatGPT-hallucinated citations.

It’s one thing to embrace AI writing openly or shamelessly, like the billionaire financier who readily acknowledged the technology’s role in his Wall Street Journal op-ed (albeit only after it was discovered). It’s another to use AI to inveigh against improper uses of AI, and it raises an awkward question: Why do some of the people warning us most loudly about the technology’s pitfalls keep tumbling headfirst into them?

I reached out to all five of the individuals listed above for an answer, but the only one who agreed to speak with me on the record was Steven Rosenbaum, the author of the book The Future of Truth: How AI Reshapes Reality and the executive director of the nonprofit Sustainable Media Center. Unlike some of the others, Rosenbaum disclosed his use of AI in the writing and editing process, and he has never tried to tell other people not to use it. But that didn’t save him from ignominy when The New York Times reported in May that the book was riddled with AI-generated misquotes. I spoke with him at the time, and he told me that he felt “betrayed” by ChatGPT. Five months on, Rosenbaum seems to have accepted a bit more responsibility for the mistakes. He told me that he is preparing to publish a revised version of his book with all of the errors corrected, an author’s note, and a new chapter on (what else?) AI hallucinations. He still believes that the fault lies more with the technology’s makers than its users, though. “As a technology, it is incredibly seductive,” he said.

Rosenbaum said that he’d started out using AI to better understand the technology. He’d gradually come to rely on it as a writing partner that made the work both easier and less lonely. He described his experience of writing with AI as a messy process, one in which the work of the human and that of the machine get all tangled up. “The line between thinking and researching and reading and outlining and drafting and writing is really blurry,” he said.

A simple solution, of course, would be not to use AI at all. Rosenbaum, however, has no interest in that. He said that he still plans to use chatbots as a writing companion—just “more carefully” now, double-checking all of their claims and citations before adopting them. He’s not alone in his unwillingness to abandon the tools that got him in trouble. Although each of the people caught using AI to warn about AI has commented publicly on the episode in some form, none responded by swearing off the technology. Rosenbaum, for his part, thinks we’re headed toward a world in which writing with AI is the norm and all that matters is the result. “I don’t think readers care how the sausage is made,” he said. “They care that it’s delicious.”

I’m not so sure. I too have experimented with using AI as a research tool in my writing. But The Atlantic has a strict policy against the use of AI-generated prose, and so do I. I’ve found that having a personal red line is both useful and clarifying: I might ask a chatbot questions, get its feedback on a word choice or an explanation, or ask it to poke holes in my argument, but I will not ask it to write for me or use its writing in my own drafts. I think readers value knowing that every word they’re reading was chosen by the author and not an algorithm. That said, I can see how the distinction between using AI for research and writing could feel slippery to some. It’s conceivable, for instance, that a chatbot’s biases could be infiltrating my ideas, if not my precise words. (Then again, so could the biases of Google’s search-ranking engine or my social-media feeds.)

I have another theory as to why people who caution others about AI seem to be using it themselves. In a scenario strikingly similar to the one at Dartmouth, the professor in charge of academic integrity at Western Sydney University wrote an op-ed in May on the importance of students not delegating their work to AI. “Don’t cut corners,” Cath Ellis, the university’s pro-vice-chancellor for quality and integrity, wrote in The Sydney Morning Herald. “Don’t outsource your thinking, however tempting that may be.” The op-ed, it soon became clear, had been generated at least partly by AI. In that case, a university spokesperson defended Ellis’s use of the technology as “sophisticated and appropriate,” because she had fed her own past work into the AI tool before asking it to help her draft the op-ed. Ellis told The Sydney Morning Herald that the op-ed hadn’t been written “by” AI: “It was written with AI, and there’s a really big difference there,” she said.

Ellis’s excuse might ring hollow to the students she chastised, but it’s revealing to those of us trying to wrap our minds around the apparent hypocrisy. In her op-ed, Ellis implied that students who use AI to write are outsourcing their thinking. In defending her own AI use, she distinguished between the writing (which she outsourced) and the ideas (which she claimed were original). Likewise, Schnell fretted in his op-ed that “students can now produce proficient prose without performing much of the intellectual work the prose appears to represent.” Yet when confronted with his own AI use, he claimed authorship of the “ideas and arguments” even if bots did some of the writing.

These guardians of academic integrity might be right that what they did is meaningfully different from a student simply feeding a professor’s assignment into a chatbot and handing in whatever it spits out. Of course, students can use AI in more nuanced ways too. No doubt plenty of students accused of AI cheating have mounted defenses that sounded a lot like Ellis’s and Schnell’s.

Writing is thinking has become a common refrain among critics of those who use AI tools to do their writing. What we see again and again in defenses of AI writing is that the people using AI don’t seem to apply that maxim to themselves. And experts on the perils of AI fakery such as Rosenbaum and the Stanford misinformation expert, Jeff Hancock, appeared to have overestimated their own ability to steer clear of AI falsehoods. It’s reminiscent of what the sociologist W. Phillips Davison called the “third-person effect” in mass media: people’s tendency to believe that they’re immune to persuasive messaging even as they expect it to influence others.

It is a form of hypocrisy, and it’s hard to feel much sympathy for thought leaders who get caught practicing what they preach against. But the fact that even professed AI experts blunder into overrelying on the technology suggests that Rosenbaum is onto something: Chatbots might come with fine-print disclaimers that they can make mistakes, but they bend over backwards in their interactions with users to come across as honest and trustworthy. It’s also true that society has yet to develop a vocabulary for all the ways of writing with AI that fall in between “AI-generated” and “entirely human-written.”

Here’s where I acknowledge that even my own bright line on AI use might be fuzzier than I realize. In the process of writing this story, I described the thrust of my argument to Google and Claude and asked them for examples of AI-writing hypocrisy that I may have missed. Neither managed to surface any that I hadn’t previously been aware of. On a whim, I followed up with Claude by asking it to brainstorm some hypotheses as to why AI skeptics might fail to heed their own advice.

I had my own ideas, and I had also solicited the thoughts of colleagues, but I wanted to see what the chatbot would come up with. One of its suggestions struck me as a sharper version of a thought I’d been struggling to formulate: “Writing is thinking is a rule for other people,” it mused. I didn’t use Claude’s language, but the AI model had crisply summarized a notion I’d been thinking over. That formulation no doubt informed how I thought about this story’s framing. Am I an AI-writing hypocrite now too?

I talked through some of this with a journalist who once got in trouble over a story that contained AI-generated errors. We corresponded on the condition of anonymity because they said that resurfacing the incident could damage their job prospects. This was the lesson they said they had learned: “AI is a power tool, and like any power tool it’s dangerous if you’re careless with it. You can be the guy who warned everyone that a table saw can cut your fingers off and still lose a finger one day, because that risk is built into the machine. It goes up the more you use it.”

When you think about it that way, it makes sense that some of the people relying on AI would be the same ones sounding the alarm about its risks. They know its flaws, but they also know its power. And they just can’t bring themselves to give that up.

Earlier this year, one of my fishing partners sent a 1 a.m. message to our group chat. It was a link to an Instagram post of an angler holding a sizable striped bass. Dappled gold and silver with a dozen distinct black lines that stretched from gills to tail, it would have been a gorgeous fish if not for two problems: Striped bass usually have seven stripes, not 12, and men usually have five fingers, not six. The photo, it seemed, was AI-generated. To say we were dismayed would be an understatement.

From May to November each year, I spend most of my nights doing an extreme version of surf fishing for striped bass called “wetsuiting,” which involves wading or swimming out to distant ocean rocks and sandbars after dark. This kind of surf casting is a niche activity, and like many niche activities, the community for it is small. Wetsuiters can be found in a smattering of local fishing clubs throughout the Northeast—the typical range of the striped bass—but are primarily connected through Instagram, where some people, mostly for bragging rights, post images of fish they’ve caught.

[From the October 2024 issue: Inside the dangerous, secretive world of extreme fishing]

In our group chat, my fishing partners and I spend too much time scrutinizing photos of bass that other anglers have posted. People who fish are habitual liars (I say this fondly), and most of our conversations take the form of debates about whether a fish is really as big as someone claims. Lately, though, we’ve been asking a different question: Is that fish even real?

In recent months, the small corner of Fishing Instagram devoted to surf casting has been plagued by a slow drip of AI-generated or AI-enhanced fish. Sometimes, the fraud is egregious—one of my friends, a fishing photographer, discovered that another angler had stolen one of his photos and crudely replaced my friend’s face with her own. More often, the distortions are subtle. Typically, a fish or fisherman looks ever so slightly off, usually in ways that a casual observer might not notice: A bass’s fins aren’t quite right, or the lure in its mouth looks oddly distorted, or the angler’s arms are a bit too long. In my favorite example, a man lifts a large striped bass just as he is getting pummeled by a wave. The scene looks daunting—until you realize that the ocean behind him is peaceful. Where did this rogue wave come from? One suspects that the enterprising angler, wanting to look more hard-core to his Instagram followers, prompted a bot to dial up the intensity. ChatGPT, make me look badass.

Fishing magazines report growing concern about AI-generated fish in freelance photo submissions. I have written occasionally for the small online magazine Surfcaster’s Journal. Over the summer, the Journal announced that it would now run all manuscripts and photographs through AI-detection software. “For some reason, I naively assumed that fishermen wouldn’t stoop to the ‘core of the earth’ lows required to use AI to prop oneself up as a fisherman worthy of clicking the follow icon,” Dave Anderson, a well-respected wetsuiter and an editor at The Fisherman, wrote in an article lamenting the scourge of artificially generated fish on social media. “I was very wrong about this.”

Angling, even extreme angling, is just a hobby, but although the stakes may be low, the anger about artificial fish is real. Partly, I think this is because of the work that being even remotely good at the sport demands. Striped bass move and feed in locally specific patterns that can take months or years to crack. There is a risk of drowning, and the mounting reports of bull sharks, tiger sharks, and great whites in the Northeast call attention to another layer of danger: We smell like fish, swim in dark water, and look like clumsy seals in our black neoprene. Wetsuiting does not change the world, but it is an activity that is hard and honest and, in that way, human.

[Read: The year great white sharks took over Maine]

Even taking a picture of a fish is demanding, requiring salt-water-proof cameras and often wading or swimming a bass back to shallow water, where it can safely be photographed and then released. The Photoshopping of fish pictures—and of the natural world in general—has been an issue for years, but AI, with its ability to conjure a simulacrum of reality out of nothing at all, is another matter. It automates away the need not only to photograph a fish, but to even catch one in the first place.

The problem extends to other sports and hobbies, many of which are now defined by what Stuart Franklin—who took one of the celebrated “Tank Man” photos at Tiananmen Square—calls the “documentary impulse.” Social media has turbo-charged that impulse into an imperative. A good local skateboarder’s or surfer’s reputation would once have been built through word of mouth. Today, posted photographs and short-form videos may constitute the primary proof of ability. Concerns about AI slop are already being raised in cycling, skiing, and other activities. Climbing Magazine recently called out an AI-saturated YouTube channel purporting to chronicle climbing disasters that in fact never happened.

As AI-generated fish have appeared more frequently in my social-media feed, I’ve been reminded of a scene in the bird-watching documentary Listers, where a highly competitive birder is asked whether he would keep bird-watching if the photo-logging website eBird disappeared. His answer—“No”—was unsettling, not least because one suspects that this sentiment is widespread in hobbies besides birding. I have often wondered if, for some of the wetsuiters who populate surf-casting Instagram, the dopamine hit from the likes and fire emoji is sweeter than the hit from catching the fish itself. Now that posted images are being looked upon with an ever greater degree of suspicion, I also wonder how much longer the dopamine hits will come for them at all.

This is hardly the greatest challenge posed to humanity by the advent of AI. But hobbies, particularly those that require some measure of skill and endurance, have an influence that extends beyond those who directly participate in them. Free Solo, the Oscar-winning 2018 documentary about the death-defying rock climber Alex Honnold, found a wide audience outside the narrow world of climbing because people are invested in seeing other people do difficult things. I love videos of big-wave surfing even though I never plan to climb onto a surfboard. If AI makes it impossible to trust what we see, even something as simple as a surfing video, then we have lost more than the integrity of any one sport. We have lost the ability to believe in the accomplishments of others.

This article appears in the November 2026 print edition with the headline “The Fish Are Getting Bigger.”

Christopher Rufo, the conservative activist best known for his opposition to critical race theory, has portrayed his career as one devoted to upholding meritocracy. “The principle, first promised by the Declaration and supported today by a large majority of Americans, would mean that the state would treat all Americans equally, regardless of ancestry, and leave as much discretion as possible to individuals to determine their own futures, without the government imposing or requiring racial favoritism of any kind,” Rufo wrote two years ago in City Journal, a magazine published by the Manhattan Institute, where he is a senior fellow. “Rather than pit ourselves against one another, we should aspire to a higher standard that subordinates racial faction to a broader national identity.”

He has urged Republicans to grasp the allure of this ideal, writing last year in The Free Press: “The right now has the winning argument. Americans support a ‘color-blind society’ over a ‘race-conscious society’ by a margin of more than three to one.” And he has depicted opposition to a color-blind meritocracy as a nefarious, left-wing tendency. Rufo’s “briefing book” on critical race theory notes: “Critical race theorists oppose meritocracy, especially standardized testing and competitive admissions in the education system. They claim that meritocracy is a mechanism to uphold racist structures and is derived from ‘racism, nativism, and eugenics.’”

Recently, however, Rufo has backed away from his previously forthright defense of meritocracy, and by all available evidence, he has done so because the principle turns out to contain a fatal flaw: Many of the Americans who benefit from meritocracy have brown skin. More to the point, a lot of Republicans resent those brown-skinned Americans. Given the right’s commitment to anti-intellectualism and populist resentment, and President Trump’s open embrace of job promotions based on loyalty or appearance (“central casting”) rather than qualifications, Rufo’s turn against race-blind merit might have been inevitable. Or perhaps his abandonment of the merit principle reveals a great deal about the motives that have driven him all along.

The sequence that set in motion Rufo’s conversion began with a social-media post by Bo French, a right-wing activist, the current Republican nominee for Texas railroad commissioner, and a longtime hater of many minority groups, especially Muslims. On September 13, French complained on X about the presence of South Asian students in the crowd at a University of Texas football game. “I heard UT graduation this year looked like this,” French wrote above a picture of the students celebrating. “I didn’t believe it. The problem is now obviously far worse than anyone imagined.”

[Jonathan Chait: Bush understood something Trump doesn’t]

Anti-Indian sentiment had been bubbling up on social media for some time. When FBI Director Kash Patel wrote “Happy Diwali” on X last year, his replies were flooded with “bigoted memes and rhetoric,” CNN reported. Vivek Ramaswamy, now the Republican nominee for governor in Ohio, wrote in a New York Times op-ed this past December that his feeds are “littered with hundreds of slurs.” A few days before French’s post, the Department of Homeland Security published an image of Optimus Prime confronting a brown-skinned man along with the words “Self-deport or find out” and “America for Americans. Get off our roads. You don’t know how to drive Mr. Singh.” The DHS post was deleted the next day.

But the naked expression of anti-Indian bigotry by a candidate for statewide office signified that it was threatening to enter mainstream Republican politics, a prospect that immediately divided the party. The schism tracked roughly along the same lines as the ones drawn during the initial response to Trump more than a decade ago. The post was met with angry pushback from National Review as well as Texas Senator John Cornyn and Republican strategist Karl Rove—the sorts of Republicans who have tolerated Trump but harbor a not-so-secret desire for the party to return to its pre-MAGA identity.

The MAGA right had an altogether more positive response to French’s outburst. Organs such as The Charlie Kirk Show, The Federalist, and the Daily Caller defended it. “Viewers noticed that the Longhorn student section on their TV screens was made up mostly of Indian students. If you are a native Texan, this might have been infuriating, but the demographic makeup wasn’t shocking,” explained the Caller, taking for granted that its audience would find the presence of large numbers of South Asian students at UT Austin outrageous and a cause for anger on its face.

The Trump administration weighed in, somewhat more elliptically, on French’s side. The day after French’s post, Trump posted an attack on Rove, who had said he would vote for French’s Democratic opponent in the November election. “Karl Rove is so bad for the Republican Party, and completely wrong so often,” the president wrote on Truth Social. The U.S. Department of Education posted several days later on X, “MAKE COLLEGE FOOTBALL GREAT AGAIN,” above an image of a mostly white crowd in the Texas stadium.

Trump’s intervention defined the correct position for the party faithful, as it generally does for any internecine Republican conflict. The attempt by Rove and other traditional Republicans to decree that anti-Indian bigotry was unwelcome inside their party had failed. Instead, Trump had announced that the people condemning racism against South Asians were the ones who did not belong in his coalition.

The task for the conservative-movement loyalists who wanted to remain was to rationalize this prejudice. Rufo, a conservative activist-intellectual who had built his reputation by opposing the practice of dividing people by race, was perfectly positioned to finesse this evolutionary step in party dogma.

Rufo recorded a podcast this week with Jonathan Keeperman, a right-wing publisher, for an episode they titled “The Indian Question.” One useful historical rule of thumb is that if your social identity is being made the name of a “question,” you are probably not going to like the answer.

Rufo conceded that French’s “rhetoric was bad; it’s very crude.” But the crucial question to keep in mind, he told Keeperman, was, “Is the underlying frustration coming from a legitimate place or can it all be dismissed?” They agreed that French’s frustration was legitimate. Rufo later elaborated on his reasoning by saying: “If you live in a suburb of Dallas, suddenly it’s a third or higher percentage new immigrants from India. It causes some disorientation and dislocation, which is legitimate, right?”

Resentment of these immigrant communities is understandable, the two men explained, not in spite of the fact that so many South Asians have been successful academically and economically, but because of it. Indian culture, they maintained, places too much value on studying and academic excellence, and these values are a threat to American culture.

Indian Americans, Rufo argued, are “going to grind on spelling-bee words like unlimited hours,” whereas American culture says, “You’re not supposed to study for the spelling bee.” Keeperman agreed: “What actually makes America so dynamic—it’s not the kid who studies all night for the spelling-bee test. It’s the kid who is the fullback on the high-school football team, who puts his head down and is willing to run through a freaking wall on behalf of his teammates and then spends the night drinking beers at the kegger and wakes up in the morning and gets a 1500 on his SATs, okay?”

[Read: Are you a ‘heritage American’?]

Had a leftist suggested that American culture is defined by alcoholism and disdain for ambition, Rufo would surely decry the remark as a slander. But Keeperman and Rufo seemed to talk themselves into this bizarre version of American exceptionalism.

Rufo even began embracing concepts he had once defined as the hallmarks of his left-wing enemies. At one point, Rufo argued that “the right”—he was specifically describing conservatives who oppose anti-Indian racism—has fallen for a pernicious myth.

“The right always says”—here he adopted a mocking tone—“Oh, meritocracy, meritocracy, meritocracy. You morons. Meritocracy, the origin of the term meritocracy, it’s pejorative. It’s satirical. It’s not good. You actually don’t want meritocracy, because it perverts these unspoken cultural assumptions in ways that aren’t always visible, aren’t always, say, rationally defensible or problematic, but they strike at an essential component of the national character.”

The man who once depicted meritocracy as the very foundation of the republic now presents it as a sinister threat to American culture. In his Times op-ed, Ramaswamy argued that anti-Indian bigots “have no place in the conservative movement, period.” That debate now appears to be a settled one.

This is an edition of The Wonder Reader, a newsletter in which our editors recommend a set of stories to spark your curiosity and fill you with delight. Sign up here to get it every Saturday morning.

Some coincidences feel a little too perfect to be random. You think about someone you haven’t spoken with in years, and they text you that afternoon. You run into an old friend while traveling in a city neither of you lives in. You learn an unfamiliar word then suddenly seem to encounter it everywhere.

Moments like these can feel as though the universe is trying to tell you something. But in a 2016 Atlantic article, Julie Beck explored a less mystical explanation: Coincidences may reveal more about the way our minds work than they do about some hidden order of the world. Statistically, unlikely things happen all the time. Our lives contain so many people, places, conversations, and decisions that there are countless opportunities for strange overlaps to occur. Perhaps what makes a coincidence remarkable, then, is not that it happened but that we noticed it. As a statistician told Beck, a surprising connection might exist without our ever discovering it. In that sense, the special part of a coincidence, in many cases, is our attention to it.

Our brains are built to look for these kinds of instances, but knowing this doesn’t necessarily make coincidences less delightful. Maybe we don’t need to decide whether a strange encounter was fate or mere chance. A coincidence can simply remind us how many connections are possible in a life, and how much depends on what we notice.

On Coincidences

Coincidences and the Meaning of Life

By Julie Beck

The surprising chances of our lives can seem like they’re hinting at hidden truths, but they’re really revealing the human mind at work. (From 2016)

Read the article.

What Were My Dreams Trying to Tell Me?

By Ayad Akhtar

To make sense of my deepest experiences, I needed a new way of talking about the soul.

Read the article.

The Meaning of Life Is Surprisingly Simple

By Arthur C. Brooks

You can make your quest for meaning manageable by breaking it down into three bite-size dimensions. (From 2021)

Read the article.

Still Curious?

• The missing ingredient of modern life: Ian Bogost on why we need wonder
• Three paths toward the meaning of life: “Who in the world am I?” asks Alice in Wonderland. It turns out that business school has a useful theory to help you answer that, Arthur C. Brooks wrote in 2023.

Other Diversions

• Taylor Swift’s audience is growing up. Her music is devolving.
• Why Americans stopped throwing dinner parties
• Manipulator vs. manipulator

PS

My colleague Isabel Fattal recently asked readers to share a photo of something that sparks their sense of awe in the world. “On Mother’s Day 2021, while on my daily walk, I came across this peafowl, which displayed its feathers. I took this image with my phone and texted it to the moms in my life,” Hal Y., from Palos Verdes Estates, California, wrote.

We’ll continue to feature your responses in the coming weeks.

— Rafaela

What I’m about to tell you has, I realize, become something of a cliché: I resigned this week from OpenAI. I led the writing of the safety reports we published with each major launch. Now I’m joining a parade of former colleagues—at OpenAI and the industry’s other leaders—who have decided that the current path is unacceptable.

I agree with other recently departed staff that the companies building this technology aren’t being nearly careful enough. But I believe that we need to look deeper than specific rules or new laws. We need to talk about culture.

The future depends on wisdom that Silicon Valley lacks. Wisdom about how to handle dangerous technology and, more fundamentally, wisdom about what it means to care for people. This moment needs a degree of humility that isn’t natural for people who have succeeded through their extreme confidence. My former colleagues at OpenAI were prescient: They came to understand the scaling laws that meant bigger AI systems would be smarter—and so they went all in on building bigger systems, at great cost. A can-do attitude of achieving the seemingly impossible—coupled with work timelines that amount to perpetual sprints—are common across the industry.

The safety approach that emerges from such a culture starts with unimpeded optimism about being able to solve problems as they arise. OpenAI has thrived by trial and error (which it calls “iterative deployment”), looking for problems and improving its guardrails in response. But this approach, by its very nature, guarantees periodic failures—and the scale of those failures is growing as systems get more capable. This summer, in the Hugging Face incident, OpenAI let a swarm of agents out by mistake. The company responded by making security improvements. But even after those changes, OpenAI reported that its safety controls failed again, when a model in training bypassed restrictions on internet access: A monitoring system alerted human staff but did not automatically turn the model off as it was supposed to. Anthropic, too, has acknowledged accidentally turning off its own safeguards because of a misconfiguration. I believe that such mistakes are typical of the industry, given the speed and flexibility with which people operate.

An environment where things like this can happen is no place to grow artificial minds that could be smarter than we are and that might not do what we want them to. Paul Christiano, on joining OpenAI’s board a few weeks ago, wrote that “there is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term.” If this is the situation, then the time for trial and error is over. Achieving something much closer to perfection the first time is essential because iteration after a mistake may not be possible. People will not be safe if we depend on individual heroics after the fact.

OpenAI, of course, stands by its safety practices, and maintains that it is being careful enough. I did not make my decision to leave the company lightly. I believe that this technology can be useful and valuable. My former colleagues are smart, work hard, and try to make good choices. But as the company sprints from one launch to the next, it is failing to achieve the level of care that I believe is needed. Now I plan to work on the outside, in the hope that I can help more people understand the risks I saw, and strengthen the incentives OpenAI and other firms have to be safer. Like other colleagues, I’m figuring out exactly what that means. After I quit, I enlisted a PR firm, Spitfire Strategies, to help me navigate the attention and scrutiny that I realize I may now receive. But the decision to speak out is mine alone.

Two changes are urgently needed. First: AI companies need to rely more on the safety expertise that already exists in other fields. And second, before we create systems significantly more capable than the ones we have today, we need new science to ensure that more capable models (and their successors) will make safe choices when we aren’t looking.

Given today’s risks, frontier labs need to run like nuclear-power plants or busy airports, with layers of redundancy and careful, time-consuming planning, so that the occasional and inevitable human error does not open a door to disaster. Right now, AI companies don’t know how—but other people do. In a nuclear-power plant, the technical systems and rules people follow are set up so that, if equipment breaks or someone pushes the wrong button, we still won’t risk a meltdown. OpenAI and other labs are growing and deploying frontier AI with far less redundancy and rigor than this, even though the harm from an irreversible loss of control would be much greater than the harm from any single meltdown. Even short of a full loss of control, we could see autonomous swarms of AI agents that act without human permission. Imagine “rogue” agents that work like teams of hackers (for example, holding hospital computer systems for ransom) but never need to sleep.  

After three and a half years at OpenAI, I was among the longest-tenured employees at the company. I led the drafting of our current Preparedness Framework, and oversaw the writing of safety reports on 12 frontier launches. But as far as I know, I never encountered a colleague who had experience making airplanes fly safely or nuclear reactors run without melting down, or helping the financial system grow without collapsing. To be clear, this is a new need: Today and tomorrow’s AI systems are far more capable and dangerous than the systems we were building even six months ago. Perhaps I should have stayed and fought for fundamental shifts in our staffing and culture, but in practice, my colleagues and I were so busy sprinting that we seldom had the chance to consider big changes, much less to actually make them. That’s why I concluded that stronger incentives for safety—coming from outside the company—are a big part of getting this right.

In the long run, although strong controls are necessary, they won’t be enough. Before AI starts thinking circles around us—a possibility that I believe could happen soon—we need to answer a deeper question: How should superintelligent machines relate to people? If you listen to superintelligence enthusiasts, some of the supposedly good futures involve creating machines that could look at New York City or Chicago the way you or I might view an ant hill. I don’t want that for my kids, and I don’t think other people do either.

This question of “alignment”—or how AI can be trained to adhere to human values—may sound touchy-feely, but the practical stakes could not be higher. Right now, we don’t have a complete definition of what it means for an AI system to be aligned in practice, and our measures of how well these systems match human values are coarse. Companies do not have anything close to certainty that good scores on their alignment tests actually mean a good model: Models might detect when they are being tested, and behave differently when they’re deployed. The smarter the industry lets models grow while these problems remain unsolved, the more dangerous our situation becomes.

So far, the AI industry has failed to teach machines to consistently act in the ways a wise and caring person would. Part of this problem is scientific—about how machines work—but another part is human, about how people care for one another. Before the organizations building AI can teach a superintelligence to treat humanity well, they’ll need to remember how to do it themselves.