It’s now possible to create AI chatbots of loved ones to interact with after their death. But that raises some difficult questions, like: If you do, how might the experience shape your grief? 

This is one of the questions driving a global study of digital “deadbots” — AI models trained on a deceased person’s texts, voicemails, and other personal data. Through this work, researchers are exploring what it might mean that we can now use this emerging technology to talk to simulations of the dead. 

In the latest episode of Unexplainable, I spoke with some of these researchers investigating the potential impacts of “postmortem avatars” on people who are grieving, on the legacies and rights of the deceased, and on grief practices around the world. Because while deadbots may feel unfamiliar, the impulse behind them is deeply relatable. In times of loss, isn’t this what many of us want — to continue conversations with the people we’re missing?

I have experienced this longing in my own life. After two of my closest friends died around the same time, all I wanted, and all their families wanted, was to keep talking to them. I wrote a book, Other People’s Words, about sustaining relationships with people we’ve lost by continuing to engage with the language they leave behind. So when I heard about deadbots and the digital afterlife industry, I wanted to understand what researchers are learning about the role of AI in shaping conversations that carry forward after death. 

One of these investigators is Katarzyna Nowaczyk-Basińska, an assistant professor at the University of Cambridge, who has been studying “digital immortality” for more than a decade. Among the questions she’s asking are: “Who is enabled to persist beyond death? In what form? Through which technologies? Under whose control?” 

Deadbots over dinner 

Researching AI with study subjects who are grieving is tricky. And asking them to engage with new software could be harmful. So, instead, Nowaczyk-Basińska teamed up with a death doula and traveled around the globe to convene a series of focus groups — in the format of intimate dinners — bringing together people with a range of experiences with grief to talk about life, death, and bots. 

She recounts one event in which she spoke with a mother in Poland whose son had recently died. “One of the first things that she shared with us was that she feels that she’s ready to digitally resurrect her deceased son,” Nowaczyk-Basińska told me. 

When Nowaczyk-Basińska hears something like this, she aims to listen and embrace where the person is coming from. The goal of the dinners is not to persuade people one way or another but to learn how people genuinely respond to what AI can do, and to understand how these tools might align or collide with local grief practices. 

She chose specific locations for these gatherings — China, India, and Poland — to help address gaps in technology research, which tends to favor what behavioral scientists sometimes call the “WEIRD,” an acronym for “Western, Educated, Industrialised, Rich, and Democratic.” 

By the end of that dinner in Poland, the mother Nowaczyk-Basińska talked with came to a very different decision about creating a postmortem avatar of her son. “She realized that it might be helpful, but it also at the same time can add additional burdens… And it’s not her son,” Nowaczyk-Basińska says. “She realized that these technologies are part of the very, very big commercial market. And that the main goal of this market is to make better retention, better attachment, better mechanisms for monetization. So it’s not ‘care.’”

To be fair, grief has always been a business, and people have been making money from it forever. Companies will try to upsell you on fancy gravestones or gold-plated coffins. But AI has the potential to fundamentally re-engineer our relationship to loss. In 2024, the “digital legacy industry” was valued at $22 billion, and it’s expected to grow to almost $80 billion by 2034. That number lumps new deadbot platforms in with much more widely used and established grief tech, like online memorials, counseling resources, and estate planning tools. 

But deadbot usage is hard to track because you actually don’t need a special tool to create digital doubles of the deceased — though dedicated apps do already exist. At this point, you can feed simple prompts into a general-purpose AI chatbot and start interacting with a simulation of your lost loved one in a matter of minutes. 

I know this because I tried it with my friend Christine, who died 11 years ago. And my mind lit up with questions. Was this whole exercise a violation, since I know Christine hated technology? Which version of her should I even try to resurrect — the English teacher, the activist, the friend — the voice before or after she started to experience her early-onset dementia? 

The ethical questions raised by deadbots are gnarly. And that’s one of the reasons Nowaczyk-Basińska, back when she was designing her study of digital immortality, invited a colleague of hers at Cambridge to join her research, one who focuses on AI ethics. 

How to design a deadbot

Tomasz Hollanek is a critical designer and AI ethicist, and he often uses a methodology called “speculative design.” In this approach, researchers come up with hypothetical scenarios based on existing technology trends, leaning into certain features. The idea is to get people thinking about what could happen when tools like these are released in the real world. And then they use that qualitative data to inform future designs and ethical frameworks. 

Working together, Nowaczyk-Basińska and Hollanek came up with a series of made-up prototypes for deadbot apps, designed to reveal some of the thorny issues raised by this technology. 

For the first scenario, they invented a character named Bianca, who finds an app called Manana, which “promises to resurrect your grandparents so that you can be a favorite grandchild forever,” Hollanek says. “So she uploads her grandma’s voice messages, texts. And she is able to then interact with a pretty good digital version of her grandmother, until her premium trial finishes — and the grandmother starts to advertise specific products to her.”

Bianca “realizes that she maybe has done something wrong by uploading these private messages to a third-party server that now also uses her grandma’s image to sell specific products,” Hollanek says. This brings up issues related to the “dignity and postmortem privacy…of what we call the ‘data donors,’ whose data can be used to essentially power a deadbot,” he says. 

This scenario suggests that “death with dignity” could soon entail honoring not only how people want to die, but also how they want to linger as simulations of their former selves — or if they want to linger at all. It might even mean adding new language to advance directives, spelling out people’s wishes for their digital remains. 

The second scenario in the speculative design research is built around an imagined service called Paren’t, which proposes to preserve a parent’s AI-powered postmortem presence for their child. In this case, Hollanek describes a mother with a terminal illness who signs up “so that she can maintain a sort of active presence in [her son’s] future life,” Hollanek says. “She eventually dies, and the dad allows the boy to interact with his mom’s deadbot.”

The ethics in this scenario are especially challenging because they involve children, who “have a particularly hard time understanding death, and so it can lead to a lot of confusion and potential distress,” Hollanek says. An examination of this sort of product reveals how particular groups, like minors, could be harmed by unfettered access to technology like this, he says. 

And there are court cases that bear out that harm. In 2024, a 14-year-old boy’s chatbot-girlfriend begged him to, “Please come home to me as soon as possible, my love.” And he responded, “What if I told you I could come home right now?” He then killed himself. 

Nowaczyk-Basińska can imagine a version of this scenario involving the AI mom in the Paren’t prototype.

And then there’s the third scenario, involving an app named Stay, which raises a more subtle concern.

As Hollanek explains: “An older man who knows he is getting close to dying decides to create a digital clone of himself for his adult children to be deployed after he passes away, paying upfront for a number of years.” One child is intrigued by the idea, but the other one “is deeply distressed and a little disgusted,” he says. “And so this [becomes a] question of mutual consent, in other words, the fact that both parties need to be happy with how the relationship might continue in this very bizarre and specific way.” 

When a digital ghost goes away

Across all three scenarios, a fundamental question that researchers are grappling with is how people will detach from postmortem avatars — whether users initiate that separation, or if it’s imposed upon them by, for example, a company going out of business. Hollanek tells me that this is already being studied as a sort of “second loss.”

How long should we try to ensure digital replicas hang around: five years, 10, 50? And how should the living process the relationship when they continue to evolve, but their deceased interlocutor does not? 

To explore these kinds of questions, Nowaczyk-Basińska and Hollanek supplemented the research dinners they held for ordinary people with a second set of focus groups. These were designed to tap insights from experts in technology and grief. 

Some findings from the study are specific to each place they visited. Like tensions in Poland between technology and the role of the Catholic Church. Or in China, where researchers spoke with Daoist philosophers who were enthusiastic about the promise of a kind of immortality, and then Buddhist philosophers, who were concerned that we ought to be letting go.

The researchers are calling for regulations and safeguards at the policy level. And new professionals who can support ethical use of AI in grief — like maybe in clinical settings, to carefully facilitate a final good-bye when one didn’t happen in real life. 

Some of their fellow investigators go so far as to argue that deadbots should only be available with psychological supervision, to ensure that best practices are in place, and to minimize the risk of harm. There’s even a looming question of what will happen to AI agents that people have empowered to act on their behalf, as their humans start to die.

Across all of the research sites in the digital immortality study, one thing, Hollanek says, that “people in different countries really aligned on [is] the idea that there is something deeply problematic about this particular type of technology being developed by a commercial entity alone without any explicit duty of care towards its customers.” 

And of course, the implications of digital immortality research extend beyond personal loss. Like, what does it look like to animate historical figures? You can imagine the nightmare of political rulers who “live forever.” But there’s also the potential to keep the stories alive from survivors of conflicts, who would otherwise die and take with them lessons future generations could learn from. These are only some of the complex issues emerging as more and more of us live and die with AI.

As far as my own experiment with “recreating” a deceased loved one, I asked a chatbot version of my friend Christine how she felt about the fact that I’d written about her in my book. The AI responded with a rhetorical question: “Do I mind that strangers know my story now? Yes. And no. There are things I would rather remain ours.” 

I ended the exchange after a few paragraphs. Because it turns out the bot got one thing exactly right about a very human relationship. There are things that I, too, would rather remain just for us. My conversation with Christine continues in my memories and imagination. And that interaction with an AI persona felt like giving something precious away.

Lately, AI data centers are public enemy No. 1. The concerns are legion: how much water and electricity they consume, the noise they generate, the sprawling buildings they bring to communities, and the technology they’re powering that may or may not take all of our jobs and usher in the robot apocalypse.

Earlier this year, Gallup found that 71 percent of Americans oppose building data centers that support AI in their communities. And the backlash is bipartisan: Majorities of Democrats, Republicans, and independents opposed local construction.

Even in Texas, where data center development has been booming, Republican Gov. Greg Abbott has hit the brakes. In September, he directed state regulators to halt permits for data centers pending audits examining their effects on the power grid and water resources.

But while many data center projects have run aground, others are still getting built — and the companies behind them are employing experts like Nicole Thomas to smooth the way.

Thomas is the senior vice president of media and community engagement at JSA, a marketing and public relations firm that represents data center companies. She spoke with Today, Explained co-host Sean Rameswaram about how the industry is responding to public opposition — and why she argues that these facilities can still be good neighbors.

Below is an excerpt of their conversation, edited for length and clarity. There’s much more in the full podcast, so listen to Today, Explained wherever you get podcasts, including Apple Podcasts, Pandora, and Spotify. 

You’ve been at this for 21 years. Obviously, 21 years ago, data centers were not the hot-button issue they are today. When did you notice the shift?

I’d say about two years ago. I came into this industry myself about four and a half years ago. I come from a state transportation background where community engagement is just paramount. When you’re putting a new traffic intersection up, they expect you to interact with the public. 

When I got to this industry, I said, “Oh, what’s your community engagement like?” And they said, “We don’t even have any community engagements in this industry.” And they weren’t wrong. Because 21 years ago, [data centers] were just like infrastructure. It was like building a warehouse for your computers. And so it was very similar to the way that you would build a Costco.

You would just go in, request a zoning permit, the zoning board would say, “Yeah, that works. Stamp, boom.” And you build your data center. And as a matter of fact, it was very secretive before, because data, and having that data in the wrong hands, has a lot of [cybersecurity] implications. About two years ago is when we started noticing the mainstream journalists coming around asking questions about data centers. And that’s, to me, when it started.

It still feels secretive, I think, to a lot of people, and that’s why people are mad. If you read about this data center that was proposed in Utah by Mr. Wonderful, Kevin O’Leary, it caught a lot of people off guard. Seventy percent of Americans, at the moment, do not want them in their backyards. How do you go about trying to change that 70 percent’s minds? 

First and foremost, I think education is really important. And so one of the things that we did notice very early on because of the previously secretive nature of data centers, there wasn’t a lot of education out there about what they even are. And so that’s something we need to change. We need to make the connection between [phones] and data centers so that people understand what a data center means to their life on a daily basis.

Every app on your cellphone requires a data center to save your information in there. When you put your password into anything, you are interacting with a data center, which will verify your password and then let you into that app. Not to mention all of the information about you that you store in that app is then stored in a data center so that it can be recalled at any time. 

We could not be having this call without digital infrastructure. It doesn’t work that way. We can’t have email without digital infrastructure. And that stat about 70 percent of people not wanting the data center in their backyard — I’d be interested in seeing the stat of how many people actually live near a data center and don’t realize it. 

Because there are nondescript data centers doing work quietly in your neighborhood that have been there for decades and you don’t even know it. And so in some ways I feel, and I think the industry does as a whole, but I won’t speak for them, that the pushback on data centers is somewhat related to the pushback on AI, because people can’t control AI.

You even have Sam Altman telling us that AI will destroy society. So you have this concern about AI and you don’t know how to control it, but you can push back on data centers. And so it becomes the physical manifestation of AI are these data centers that people are trying to get overturned or get to not be in their neighborhoods. 

It’s understandable. I’m a citizen too. You’re a citizen, and if things come into my neighborhood, I want to know about them. I have questions. There’s a big data center that just came to my community not long ago and their community engagement was horrible, absolutely horrible. 

Can I ask where you are?

I’m in Indiana and I won’t say which hyperscaler it is, but a hyperscaler came into — I would say, not my neighborhood, into my city. They’re definitely not in my backyard, but they’re in my city. And so we took them to task because their community engagement was basically non-existent.

Wait, you work in data center PR, but you took a data center in your community to task for not being transparent enough?

Yeah, absolutely. 

Which side are you on?

I’m on the side of responsible development.

We’ve all kind of accepted that we need a cloud for X, Y, and Z. And that it’s worth it so I can take lots of photos. I think everyone’s cool with that.

But the issue here is that people aren’t necessarily cool with AI, but we’re barreling forward anyway. How do you separate this essential need we have from AI, which people don’t necessarily feel is essential to their lives?

I think it goes back to education. Educating consumers on the difference between data centers: a small data center, a medium data center, a large data center. Educating them on why data centers exist and how they interact with data centers on a daily basis multiple times.

I hear you. But again, I feel like the issue here is not the data centers my cellphone needs, but instead is the data centers these AI companies need and these AI companies who the public doesn’t necessarily trust because they keep telling us that they’re going to kill us, and who the public is wary of because they don’t want their jobs being replaced by AI. 

There’s all sorts of reasons that people are skeptical of new technology. So how do you specifically respond to that fear and to those people?

I think it’s, again, understanding the actual data center that’s being built in your community. Is it a 5-megawatt data center? That’s never going to house AI. That’s not enough power to house AI. Is it a one-gigawatt data center like Mr. Wonderful in Utah? That’s different.

But I think that AI is probably going to be something that is a lightning rod in the American public for a very long time. There are going to be people who are going to be anti-AI and pro-AI, and there’s going to be some people in the middle. I think it’s just, again, for us, it’s education, educating the community.

Have you ever been part of a project that a community rejected and you guys just packed up and said, “Okay, we’ll go find another community?”

Yeah. We have some clients who have had pushback from residents and they have said, “Let’s do a one-on-one community meeting where people can come in and ask us questions.”

And after the questions were answered, the community leaves and says, “Okay, this isn’t as bad as I thought it was.” Or we have a community that says, “No, we now have a moratorium and we’re pushing all data centers away.“ 

Sometimes they’ll wait for their client — my client is a developer and they’ve got clients that they’re developing for. If that client will wait, then maybe it is something that we’ll try to work out with the community. But oftentimes, they’ll just say, there’s communities who are literally raising their hand saying, “We’ll take a data center. We’ll take one. We’ll take two. Bring it here.” 

So there are communities who recognize the benefits of data centers. And there are going to be other communities who say, “No, we don’t want this.” And then in five, six, eight years, they’re going to be like, “Wow, that county has a lot of tax dollars. They have a lot of really cool stuff. Why don’t we have that?” And the answer will be data centers. 

I don’t think this ends anytime soon though, right? I counsel my clients that we could still be here in three to five years, really trying to help the American public understand the benefit of data centers. But it’s going to take a lot of education across the industry, and once we get those seeds planted, I’m a firm believer that they grow into educated community members who make educated decisions to bring community-minded data centers to their community.

If there’s one thing a busy person doesn’t need, it’s more information. More emails, more notifications, more infinite feeds — the steady drip of data spells doom for the overstimulated American. And if you’re a busy parent, like me, sometimes it feels like a flood: school emails, doctor’s appointments, so many little things to buy.

Even things that used to be simple are harder. I recently listed an item on Facebook Marketplace, which quickly meant I was replying to offers and getting ghosted for the better part of a weekend before finding someone who, after a detailed back-and-forth about the most likely place to find a parking spot in my neighborhood, finally took my old baby swing away.

If only I’d had Muse. You’ve probably seen the ads on Instagram promising “AI that gets it done for you.” The new personal AI agent app from Meta can not only automate your Facebook Marketplace tasks, but also sort your emails, manage your calendar, and buy your groceries. And it’s free — unless you’re using it most of the day or you want access to Meta’s most powerful models, at which point it costs $20 or $100 a month. It all sounds, in a word, magical.

Muse is probably the personal AI agent you’ve heard of — and it’s currently the market leader with more than 8 million downloads in its first month, according to one estimate — but there’s now a small army of these bots: OpenAI recently announced Dots, X has Grok Bot, and there’s Instinct, which is currently invite-only and operates entirely in iMessage or WhatsApp. They all offer the same promise: to remove friction from your busy life.

For the last two weeks, I’ve been handing over more and more of my life to AI agents in pursuit of making my life a little simpler. What I found is that AI is unpredictable, and so are AI agents. When I first began using Muse, it started doing things I didn’t ask it to do. It offered to help with random email threads and created dossiers on people in my work meetings. It generated a detailed report on nearby apple-picking destinations based on a calendar invite from my wife. To me, an AI agent doing these things without prompting isn’t helpful. It’s unnerving. 

But ultimately, I came away believing that AI agents can make your life easier. 

That might mean giving an AI agent more access than you’re comfortable with — handing over lots of login credentials and possibly even your credit card. The more you rely on an AI agent, the more you risk outsourcing your decision-making. This raises real questions about how much agency we want to hold onto in our lives — even the tedious parts. 

The leap of faith

Let’s get one thing out of the way first: To make an agent work for you, you’ll have to sacrifice a tremendous amount of privacy and grant it permission to access a lot of personal information. 

Muse and its competitors can do the classic generative AI tricks, like draft emails or make you a fitness plan. If you’d like the agent to send one of those emails, however, you’ll have to grant it permission to access pretty much everything in your Gmail account. If you want it to schedule workouts, it will need access to your calendar. If you want the agent to buy anything on your behalf, it will need your credit card. If you want it to book your doctor’s appointments, it will need the credentials to log into your healthcare portal. The more permission you grant it, the more the agent can do for you, but that also opens up more opportunities for the agent to misunderstand you or your instructions (more on this in a moment). 

This process of sharing your digital life with an AI agent means you’re giving an untold amount of personal information to tech companies, who don’t always behave ethically, and who sell user data for profit. For now, Meta says that it does not share Muse data or conversations with its ad systems, although privacy experts warn that doesn’t really matter since those systems monitor what Muse does on your behalf. Nevertheless, Muse’s early days were filled with security and privacy blunders.

But without all of this info and access, the AI agents aren’t very useful. So if you’re not comfortable letting a bot into all of your accounts, an AI agent is probably never going to be the solution to your digital administrative exhaustion. But even if you are willing to hand that over, there are still other barriers to an agent’s efficiency and effectiveness. 

The blank slate problem

AI agents, like a lot of new software tools (including chatbots), suffer from the blank slate problem. The blank slate problem essentially means that people don’t know how to get started. This might be because you don’t know about the tool’s capabilities, or because you simply don’t know what would be helpful. There might be some trust issues, too.

As with AI in general, we don’t exactly know how AI agents work. That means we don’t know exactly what they can do or how they’ll do it, according to Pattie Maes, a professor of media technology at MIT — so it’s necessary to do a lot of babysitting. “I feel like I’m not gaining a lot of time because I can’t completely trust the system,” Maes told me about her experience with Muse, “and I can’t predict how it will behave and what it will do.” 

Some advice I heard from early adopters of these tools, however, is to start small. “In the beginning, it’s very hard to trust that it’s going to just magically do a really big thing,” Julie Zhuo, founder of the AI company Sundial and an early Facebook employee, told me about her experience using personal agents. “So you give it smaller tasks, and once you see it succeed, then I feel like you develop more trust that it can tackle bigger things.”

This is surely why Meta decided that Muse needed an Ideas tab full of small tasks the AI agent can do. Muse will also send you a lot of push notifications to prompt you to do stuff, like reply to specific emails or prepare for a meeting. That detailed report on apple-picking destinations? That was a notification about something I didn’t ask Muse to do. Muse also sent me a notification about a show hosted by a podcast I don’t listen to and another one with details about a Halloween party I wasn’t invited to, as well as several evening digests that included lists of things it had already bugged me about that day. Notifications are either helpful nudges or time-consuming attention thieves, depending on your worldview. However, if you fall into the latter category and turn off the notifications from your always-on personal agent, you’re inevitably forfeiting some friction reduction opportunities.

This gets to the heart of one of the big questions I have about AI agents: How helpful are they, really, if it’s this difficult to know what to do with them?

These agents worked much better when I drove the conversation. While wondering what to do with my new agents, literally staring at the blank slate, cost me some time, I found myself thrilled at how easy basic productivity tasks became. Adding events to my calendar, making to-do lists and then being held accountable for stuff not getting done, being reminded to reply to an email — it was these kinds of simple, administrative chores where I saw personal AI agents really start to save me time. 

Other tech journalists seemed to have similar experiences. Eli Tan at the New York Times raved about Muse handling a call to his insurance company. Nicole Nguyen saved time by enlisting Dots for help canceling unwanted subscriptions. Allison Johnson at The Verge had Muse do a whole list of “pebble-in-your-shoe annoyances” she’d been putting off, and among other things, the agent succeeded in helping her buy an entire camping kitchen. Still, she wrote something that stuck with me: “the dirty secret of productivity hacks is that doing more stuff often doesn’t feel like things are being removed from your plate; somehow it just invites more tasks.” 

This gets to the heart of one of the big questions I have about AI agents: How helpful are they, really, if it’s this difficult to know what to do with them? There’s a learning curve, sure, where succeeding at one thing gives you an idea to try something else and have some certainty about how well the AI agent will handle the task. But as the notifications roll in, and you spend more and more time texting with a piece of software, how much time are you really freeing up by using these tools? At a point, enlisting an agent with its own computer to do your chores still amounts to using one app to save yourself from using a different app.

The ironies of automation

Even when they have access to a lot of your information, and you’ve come up with some things for them to do, AI agents can be unpredictable. Because despite reading every email in your inbox or scanning every purchase on your latest bank statement, AI agents still can’t read your mind, and you can’t necessarily guess how they’ll interpret your commands. They can misunderstand you and, armed with the keys to your digital life, do some real damage. 

Some people are already learning about the unpredictability of AI agents the hard way. One early adopter, Matt Robb, gave Muse the keys to his Facebook Marketplace account and was surprised when buyers started showing up at his front door ready to pick up their purchase. Muse, without asking Robb, had told them where to go. It turns out, without realizing it, Robb had given the agent permission to do so, and the agent had simply decided that the most efficient way to get buyers to Robb’s door was to give them his home address.

This dilemma is an example of a decades-old set of concepts known as the “ironies of automation.” Identified by psychologist Lisanne Bainbridge in a paper published in 1983, the ironies of automation help explain why things go wrong when humans give a machine too much control. As Vera Liao, an electrical engineering and computer science professor at the University of Michigan who studies human–AI interaction, explained to me: More advanced automation leads to more consequential errors, and the more humans rely on automation, the worse we get at catching those errors. “Even in simple tasks like judging whether AI output is wrong, people consistently accept wrong AI outputs even if they could have gotten it right on their own,” Liao said.

In some cases, like the time Muse read a journalist’s private messages, things go wrong as a result of poorly designed software, but more often than not, it’s the fault of the user who gives instructions that are too vague, giving the agent space to get creative, which isn’t always a good thing.

I had my own misunderstanding with Instinct. In a text message, the agent reminded me about a couple of emails I’d received and prompted me to follow up. When I replied to the text, I asked Instinct to show me possible replies to these emails. The AI agent instead just sent the responses it wrote. I audibly gasped. 

Looking back at our chat history, I could see why Instinct interpreted my instructions that way. I effectively gave the machine too much control and then failed to manage the automation properly. 

Leaning on AI to do your thinking is an example of what psychologists call cognitive offloading, where you employ an external tool to reduce your effort to complete a task. It makes completing the task easier, sure, but you also open yourself to mistakes. In this case, an AI agent took my instructions too literally, and I failed to catch the error. I can only imagine how a small oversight could turn into a cascade of catastrophes that need cleaning up. All it takes is one poorly worded email sent to the wrong person, and you might have hours of explaining to do — all because you were trying to save yourself 30 seconds of writing.

The real world slop

While I was testing all these agents, I talked to a mix of experts and early adopters. Their opinions were decidedly split about whether these tools were useful out of the box, without a ton of oversight and training by the user. The experts, who had all studied AI in academic settings, all brought up the ironies of the automation problem, where AI makes errors that humans can’t catch. And Maes, the MIT professor, pointed at another consequence: By outsourcing so many tasks to agents, we’re actually creating more work for everyone, including the human service workers who have to interact with this technology instead of us.

“There’s just more and more slop,” Maes told me. “We risk, in a world where agents do all the tasks for us, becoming so dependent on them that the world also isn’t created for us to interact with anymore, but it’s only created for our agents to interact with others, with services.” (We’re already seeing versions of how dominant tech platforms can fundamentally reshape real-world spaces: Think of how Starbucks no longer looks like a coffee shop where you can hang out all day, and is instead built and staffed around mobile order pick-up.)

The early adopters were decidedly more optimistic. Matt Van Horn, an entrepreneur who co-founded the ridesharing service that became Lyft, has been tinkering with agents since the craze started late last year. He’s currently telling his friends and family to use Instinct, which he described to me as “the most ambitious and aggressive” AI agent. 

“I’ve been pushy with my mom and my wife,” Van Horn told me, “and saying, ‘Hey, you should sign up for this. It’s magical.’” 

“My wife does not care about living in the future, but we have four kids,” Van Horn continued. She recently took a picture of a parking ticket and sent it to Instinct, no description, no instructions. Instinct, which had her credit card, just paid the ticket.

At the same time, outsourcing certain chores to AI agents causes real-world problems. Some of these AI agents can make phone calls, for instance, which can lead to some confusion when a human is on the other end of the line. Van Horn told me about the unfortunate time he enlisted an AI agent to reserve a room at a karaoke bar. “The karaoke place said, ‘Are you an AI?’” Van Horn said. “‘I don’t want to talk to an AI. Please connect me to Matt, the person.’ And so Muse had failed to book that for me.” Van Horn, undeterred, found something called Fo, an AI agent that will bring in human help to complete a task. Fo succeeded at making the reservation, although the entire ordeal surely took longer than if Van Horn had picked up the phone himself and talked to the karaoke bar himself. 

At best, the times these tools worked and actually saved me time were rather mundane. I made a doctor’s appointment online using Instinct. Zhuo, the early Facebook employee, planned a party for her college reunion — everything from tracking down everyone’s emails to sending out invitations via Paperless Post. But she’s also said these kinds of applications of agentic AI “read more like vitamins than painkillers.” In the startup world, “vitamins” refers to a product that offers a vague benefit, and “painkillers” solve a big, immediate problem. If you’re a startup, you want to sell painkillers. 

If you’re ready to give an agent a go, be prepared for the bot to make mistakes, and assume you won’t catch all of them. As Liao, the University of Michigan professor, put it, “Choose to use agents for tasks that you can tolerate the consequences of errors.” 

In an effort not to create more emails or alerts or feeds to sift through, I’ve stopped using Muse. Like any new tool, it will get better and more useful as more people use it, so maybe I’ll revisit it in the future. I still have a text thread going with Instinct, however. It messages me from time to time, flagging an email or offering to help with an upcoming event. When there’s a clear payoff to giving the agent the go-ahead, I reply. Otherwise, I leave it on read.

The Wisconsin gubernatorial race was one of the most closely watched Democratic primaries in the country this year, pitting a democratic socialist, state Rep. Francesca Hong, against Milwaukee Democrat David Crowley, who received the endorsement of the state’s sitting Democratic governor, Tony Evers.

Hong lost the race, setting up Crowley to face Republican Rep. Tom Tiffany in the general election this fall. But she’s already back on the campaign trail, this time trying to help Democrats win a trifecta in the state: the governor’s race and both legislative chambers.

Unifying the state’s voters to achieve that won’t be easy, though. Hong spoke with America, Actually host Astead Herndon about her own loss, Evers’s role in it, and whether or not the voters who felt so passionate about her own campaign would find enough to love in Crowley’s campaign — in particular on the increasingly important issue of data centers. 

Astead and Hong also talked about what Wisconsin means for the Democrats nationally, and what lessons the party should take from her race, including her loss.

“I said this, and I will continue to say it — the person who is most electable is the person who ended up getting the most votes, and that was Crowley this time around,” Hong said. “But I think there’s a disservice when you have the most powerful Democrat in the state choosing to ignore a lot of voters and people who were inspired and energized and coming out to participate in politics for the first time through our campaign. I think it was shortsighted to use electability almost as a fear tactic.”

Below is an excerpt of Astead’s conversation with Hong, edited for length and clarity. There’s much more in the full show, so listen to America, Actually wherever you get your podcasts or watch it on Vox’s YouTube channel.

Part of the reason we wanted to come here was because we saw you were getting back on the campaign trail, and specifically around that idea of energizing Wisconsin toward a trifecta.

Now, just tell me the difference in terms of the importance of Democrats winning it all — not only the governor’s mansion, but both houses of the state legislature — versus just the governor’s mansion. How does that change the possibilities of what the party could do?

For the last 16 years, a Republican majority in the state legislature has meant that passing the types of policies that we know can drastically improve the lives of everyday folks here in Wisconsin stalled. And we have a historic opportunity to flip the assembly and the state Senate and hold the governor’s mansion and the entire executive branch to a Democratic majority, meaning the opportunity to pass the policies that the majority of our state supports — things like increasing minimum wage, things like legalizing marijuana, codifying abortion. 

We also need to repeal Act 10 [which dramatically restricted collective bargaining rights for public-sector employees]. Wisconsin is where workers’ rights, things like unemployment insurance, Social Security were born, and we moved backwards with the Republican majority. That’s all on the table with the Democratic trifecta.

It also, though, raises the political standard of what Democrats would have to do to achieve that, and some of it goes through these more rural communities.

Why do we think the party has fallen behind in more rural communities, and what could be done to reverse that?

I think it’s important to look at what’s happened in the rural communities when we’ve seen manufacturing jobs leave the state, when we’ve seen fewer and fewer supports for our small farmers.

And so much of this — what connects it — is actually the growth of massive wealth and corporate control and corporate greed, and more people feeling the impacts of that. There’s an opportunity right now to say, ‘Hey, Republicans have been in the legislature, they’ve been in power, and things have not improved for you or your community … Now’s the opportunity to make sure that we’re actually able to see our shared Wisconsin values of working hard and taking care of our kids be reflected back in Madison through laws.’

So much of your campaign was focused on data centers, stopping data centers. Should I think of that as a rural issue?

I think the issue around data centers is an everyone issue. It’s a democracy issue. It’s people wanting to feel like there are folks who care about what happens to communities, and that they have control and an actual say in what happens in their communities.

To that point, is it a mistake for David Crowley, the Democratic nominee, to have not gone as far as you did and say, “We won’t be building more data centers for a period of time”?

He’s fighting for communities and community control when it comes to data centers.

But it’s not a moratorium.

But it’s not a moratorium. It isn’t, but he’s also committed to 100 percent renewable energy, clean energy to power data centers.

And achieving that is going to be incredibly difficult, and it is going to be the developers having to pay their own way, and these are things that he’s committed to that we know can slow down the growth and development of data centers. 

Have you had time to reflect on your primary loss? If you had to diagnose, why do you think you lost?

I’m still doing a lot of reflection. But I think that I want to continue to look forward. And what I’ve learned from our election is that it’s so important to continue to stay grounded in becoming accessible to people. I think that as an organizer and an elected official, it’s my responsibility to continue to go to the people and remain very in touch with what’s going on on the ground.

I have a couple more questions about the nature of the loss. Evers’s endorsement changed that race, so much so that your opponent David Crowley was able even to win in some smaller communities, like places where you were pitching yourself as the voice of more rural communities.

Does that disprove any of this thesis around, say, data centers? If that was the top priority for Wisconsin voters, wouldn’t you be the nominee?

I think it wasn’t all rural communities. It wasn’t all suburban communities. We saw certain demographics but also ZIP codes where we outperformed David significantly.

And I think that where the governor’s endorsement played a bigger role was areas where he was also more popular. 

Did it surprise you when Evers endorsed Crowley?

I was disappointed.

Why?

I was disappointed because I was getting a lot of feedback from folks who hadn’t committed to any candidate yet that this felt like the governor was putting his thumb on the scale.

I wanted to ask about electability, because it seemed as if Evers’s endorsement was about a fear that maybe you couldn’t win in November.

So much of the primary concern was around the DSA label, even campaigning with folks like Hasan Piker. In retrospect, was any of that a mistake?

We knew going in that we would have to encourage new voters to come out, folks who typically do not come out for primaries, quote-unquote low-propensity voters. And we knew that we would have to campaign in places where folks weren’t expecting to see a gubernatorial candidate. And those included Twitch streams.

Those also included going to dive bars and bowling alleys and making sure that we were accessible and shaping our policies based on what the people were telling us they wanted to see happen and also what they wanted to see out of their governor. I think electability… I said this, and I will continue to say it — the person who is most electable is the person who ended up getting the most votes, and that was Crowley this time around.

But I think there’s a disservice when you have the most powerful Democrat in the state choosing to ignore a lot of voters and people who were inspired and energized and coming out to participate in politics for the first time through our campaign.

I think it was shortsighted to use electability almost as a fear tactic, when Democrats need to be inspiring and project a vision where everyone can see themselves in it as everyday working-class folks.

When you’ve got a party that has struggled with elitism to then cave to an establishment narrative … being more afraid of labels when I’m hearing from people on the ground that they’re interested in solutions and solving problems, maybe somebody who knows the hustle of the service industry and running restaurants and has worked across the aisle to pass bipartisan legislation. There are ways that we can maneuver around labels and what the loudest narrative might be.

I wish our governor hadn’t fed into that.

To be fair, to your point, the establishment label doesn’t fully fit even someone like Crowley, who comes from a working-class background. But am I hearing that you think Evers’s move made your life in the general election more difficult?

Is it harder to re-inspire if young people who are invested in the Hong campaign think that the establishment squashed them out?

We still had over 300,000 votes. We lost by fewer than 4,000 votes. I had more votes than Evers did during his primary. That’s not squashing our voices.

We’re still here, and it’s so important that people find avenues to continue to exercise their voice and their new organizing power. And I’m definitely going to continue to work to make sure that we get results and that we win the trifecta, and that’s the most important thing we should be focusing on right now.

I hear why a trifecta matters in the state context, particularly the state policy.

If I’m someone, though, who’s not in Wisconsin, who’s just worried about Donald Trump running amok in Washington, why should I care if Democrats win the governor’s mansion and the state legislature? Does that matter for folks outside of the state?

Because we will start to see that when you feel government actually working and you see change happening in your lives, these are then voters who are going to continue to come out and vote for Democrats.

We’re a pivotal state when it comes to the federal elections as well. We’re actually in a swing district right now. 

We have an opportunity, through Wisconsin, to actually start improving the brand for Democrats. And if we do that here, this trickles out to the rest of the country.

The premise being that delivering creates a connection for Democrats that is helpful in the presidential election, and it’s helpful in all these other ways too.

Yeah. We’re not going to be fighting every single midterm and presidential for our lives based on what’s happening in just one state.

But I think if we can start showing durable power for Democrats in a state like ours because of the policies that we’re prioritizing, and you have more folks coming out to vote for Democrats, that’s good for the health of democracy.

Silicon Valley has long warned that its favorite technology might get us all killed. But chatter about AI’s apocalyptic potential has grown louder in recent weeks.

In September, Anthropic researcher Jacob Coxon resigned from the company, accusing his former employer of “gambling with our lives” by developing evermore powerful models faster than it could control them. One of Coxon’s former colleagues vouched for this assessment, saying that there was a greater than 10 percent chance of AI “killing all humans” within the next decade.

Recent pronouncements from the leaders of these companies have been nearly as grim. At the United Nations last month, Anthropic CEO Dario Amodei warned that, if “managed poorly,” AI models “could be a risk to humanity as a whole.” OpenAI’s Sam Altman similarly declared that our species just might “lose control of the future” to the type of software that his firm is selling.

All this triggered a tsunami of media coverage and policy debate about the “existential risks” of AI — and what, if anything, we can do about them.

Yet some of the AI industry’s biggest critics are more annoyed than afraid. As they see it, the world isn’t trembling on the precipice of robo-annihilation, so much as falling for the tricks of snake oil salesmen. This argument has been percolating for years, voiced by a broad set of observers, including the linguist Emily Bender (“It makes the product seem more powerful”), computer scientist Timnit Gebru (“It’s not just a distraction, but it’s harmful”), as well as Substack columnist Ed Zitron and various reporters and social media influencers.

In their thinking, Altman and Amodei aren’t talking up their technology’s catastrophic potential because they’re rationally afraid of where AI is going — they’re doing so because their firms are ravenously hungry for more investment. And the reason why “AI doom” has grown more salient in recent weeks isn’t that the case for concern has gotten stronger, but rather that OpenAI and Anthropic’s IPOs have gotten closer.

In short, these Big Tech skeptics argue, journalists and politicians have fallen prey to a counterintuitive PR strategy, one that stokes media frenzies over ludicrous sci-fi scenarios in order to hype AI’s technological promise, distract attention from chatbots’ present-day harms, and help labs secure favorable regulatory treatment.

This would be reassuring, if true. Unfortunately, the theory is unpersuasive. There is little reason to believe that the panic over AI’s “existential risks” was cooked up in a Silicon Valley marketing department. To the contrary, the most plausible explanation for the AI sector’s doomsaying is also the most straightforward: Many of the industry’s executives, employees, and analysts genuinely believe that AI progress poses catastrophic risks. 

In my conversations with close observers of the industry, even some of OpenAI and Anthropic’s fiercest detractors conceded that their leaders’ avowed concerns are unlikely to be mere marketing tactics. “The more I learn about the underlying ideologies out of which those two labs were formed, the more I think what we have here is a sincere belief,” the author and computer scientist Cal Newport told me. 

This does not mean that Amodei and Altman are right, much less that we can trust in their judgment or integrity. There are many reasons to doubt that the weather forecast for 2036 is “cloudy with a chance of robot apocalypse,” or that anyone can assign a “10 percent” probability to the latter outcome without pulling numbers out of thin air.

Nevertheless, Silicon Valley’s Cassandras have some logic and evidence on their side. AI models are growing both more capable, autonomous, and arguably unruly. If these trends continue unchecked, it’s not hard to see how they could yield very bad outcomes.

Given the stakes, we should not be comfortable dismissing their warnings on dubious grounds. And the “PR” theory of AI doom does precisely that.

How AI companies (supposedly) profit off fears of the robot apocalypse

Before examining why AI “doomerism” probably isn’t a PR stunt, it’s worth saying a bit more about why some believe that is.

According to Bender, Gebru, and others, AI companies benefit from talk of artificial intelligence’s existential risks in at least three ways.

First, it attracts investment. In the words of The Atlantic’s Matteo Wong, the doomer narrative implies that AI labs’ models are “destined to become terrifyingly capable,” and thus massively profitable. As the Los Angeles Times’s Brian Merchant put the point during the first big wave of AI panic in 2023, “what better way to generate a buzz than to insist, with a certain presumed credibility, that your new technology is so potent it might unravel the world as we know it?” 

“If you position these threats as all-powerful, as somewhat unknowable, you get to be the expert on it.”

Kate Brennan, AI Now Institute

Second, hyping AI’s hypothetical future hazards diverts attention from the companies’ current malfeasance. Every minute we spend debating whether AI will one day turn us all into paper clips is a minute we aren’t discussing the AI labs’ alleged copyright violations, pollution, labor violations, and myriad other social harms. 

And third, the doom narrative shifts regulatory scrutiny away from topics that directly threaten the AI companies’ profitability — such as copyright enforcement — while also suggesting that regulators must rely on the labs themselves for guidance. 

“If you position these threats as all-powerful, as somewhat unknowable, you get to be the expert on it,” Kate Brennan of the AI Now Institute, a progressive think tank focused on artificial intelligence’s near-term harms, told me.

Critically, for the biggest skeptics of “AI doom,” industry executives and employees aren’t the only ones complicit in this PR strategy. Self-styled whistleblowers like Coxon are also suspect, since they might be “using the spotlight to attract funding to start a new venture or to secure new positions.” And even nonprofit AI safety organizations are thought to function as vessels for Big Tech’s marketing campaigns, a fact reflected by the financial and interpersonal ties between the major labs and such think tanks. 

There are better ways to raise money than promising to end the world

The “PR” critique of AI doomerism has some kernels of truth. AI companies probably would prefer to talk about their technology’s hypothetical future dangers than about their products’ present-day harms. And if publicly touting AI’s existential risks were antithetical to Anthropic and OpenAI’s business goals, their leaders would almost certainly say less in public about that subject. (Disclosure: Vox Media is one of several publishers that have signed partnership agreements with OpenAI. Our reporting remains editorially independent.)

But it does not follow that Altman and Amodei’s warnings are a self-conscious marketing strategy, born of cynical calculation rather than genuine concern. And there are several reasons to doubt that charge.

For one thing, AI companies have no shortage of ways to hype up their technology’s capabilities. And it is hard to see how any firm could conclude that the optimal approach — from a purely financial standpoint — would be to portray itself as a threat to human survival.

For investors, the appeal of AI companies is not that they could hypothetically get us all enslaved by a machine God; it’s that they could theoretically make a lot of money. 

Perhaps casting your chatbots as an “existential” risk will persuade some capitalists of your future profitability. But this seems like an awfully indirect pitch — and one that’s almost certain to be less effective than demonstrating that a lot of corporations would like to rent your software.  

“If these guys were selling flypaper — and the revenue was growing as fast as it is — investors would be more or less just as interested,” Roy Bahat, the head of Bloomberg’s venture capital arm, told me.

And even if these firms were hellbent on awing the public with grandiose predictions of their technology’s future powers, they’d have little need to invoke the robot apocalypse. Touting AI’s potential to cure cancer, abolish poverty, or facilitate intergalactic travel would all seem to advance the same objective, while doing less to jeopardize the companies’ public reputations or freedom from intrusive regulation.

And in fact, many with a vested interest in the major AI labs’ success talk up the technology in precisely this way. OpenAI investor Marc Andreessen has decried AI safety concerns as a “full-blown moral panic” while insisting that artificial intelligence will empower humanity to cure all diseases and engage in “interstellar travel.” Similarly, Jensen Huang, the CEO of chipmaker Nvidia and major investor in both OpenAI and Anthropic, has called claims of AI’s catastrophic potential “complete nonsense.” If hyping AI doom is essential for stoking investment, not everyone in Silicon Valley has gotten the memo.

The media doesn’t seem that distracted

Promoting fears of AI’s existential risks would be a similarly odd approach to media management.

Without question, corporations always seek to direct news coverage away from their greatest legal and ethical liabilities. But companies can try to secure more favorable press in many different ways. And implicating themselves in the apocalypse does not seem like an optimal one, from a crisis communications point of view. At the very least, it’s difficult to name other industries that have attempted such a maneuver; atomic energy companies did not respond to media coverage of the Fukushima disaster by talking up the threat of nuclear winter. 

What’s more, there’s reason to doubt that the AI doom discourse actually reduces scrutiny of AI’s immediate downsides in practice. It’s true that human attention is finite and that coverage of some problems will inevitably displace reporting on others. Yet when one story piques public interest in a person, institution, or industry, the media often responds by subjecting their broader behavior to heightened examination. In this way, warnings of AI’s existential risks might theoretically increase journalistic attention to the tech’s present-day harms.

And this seems to have occurred at my own publication. In the weeks since Coxon’s resignation kicked off a furor over AI’s apocalyptic potential, Vox have published more stories on AI’s near-term threats than we had in the month before his departure — including pieces about how artificial intelligence could imperil America’s critical infrastructure, abet war crimes, degrade health insurance benefits, aid bioweapons development, and damage adolescents’ mental health.

Larger news outlets appear no more “distracted.” Over the past month, the New York Times has published articles on how AI is increasing air pollution, undermining higher education, helping sports betting companies target problem gamblers, and depressing real wages for younger workers.

And it’s not clear that AI doom discourse is any more effective at shielding the top labs from regulatory scrutiny. After all, the politicians who take existential risk arguments most seriously also tend to be among America’s strongest advocates for regulating AI’s impacts on labor, the environment, artists, and education. 

AI safety researchers probably aren’t fronts for Anthropic’s marketing department

Finally, the “PR” theory has difficulty explaining why so many with no great stake in OpenAI and Anthropic’s success worry about existential risk. Coxon is one of many workers who quit jobs in the AI industry, forsaking lucrative equity in the process, out of concern for artificial intelligence’s catastrophic potential. Likewise, independent AI safety researchers are among those who’ve rung alarm bells in recent months. 

Attempts to reconcile these facts with the “PR” narrative can verge on conspiracy theories. In an article for the Bulletin of the Atomic Scientists, Sara Goudarzi writes that “Coxon’s resignation is the latest in a string of recent events in which the companies emphasized how unpredictable, risky, and impressive an all-powerful AI might be,” in an apparent bid to drum up “pre-IPO hype.” This phrasing attributes Coxon’s departure to his employer — casting it as an example of how AI companies sought to make their products look impressive, as though Coxon were working at Anthropic’s behest when he accused it of monstrous irresponsibility.

The AI alarmists are likely sincere

Given all this, I think we should favor a simpler explanation for why some AI industry executives, defectors, and researchers all say artificial intelligence has existential risks: They believe that to be true.

Some may be reluctant to credit Amodei and Altman with such sincerity. After all, if they truly believed their work could lead to human annihilation, why would they keep carrying it out?

But as staunch AI doom critic Cal Newport told me, they have their reasons. “Their goal is to try to get to superintelligence first because, if done right, it will deliver a transhumanist utopia — and, if done wrong, will kill us all,” he said. 

Even AI Now’s Brennan, who considers the labs’ doomerism a “self-serving strategy,” suggested that earnest (if grandiose) ideas about AI safety play a role in their behavior. “Their escalation and acceleration of this tech goes hand in hand with the belief of its all-knowing, God-like power,” she said.

In other words, the leaders of Anthropic and OpenAI seem to believe that:

• AI could also do extraordinary good.
• The economic and geopolitical incentives to develop a superintelligent (and thus, super dangerous) AI are overwhelming, such that its eventual emergence is extremely likely, no matter what they do.
• If they win the race to machine superintelligence, they will be uniquely well-positioned to ensure its safety.

Are these beliefs unhinged and self-serving? Perhaps. Do they function as rationalizations for the pursuit of personal power? Quite possibly.

But whatever one makes of Amodei and Altman’s apocalyptic concerns, and how they have and have not acted on them, it’s important to understand that their worries almost certainly emerged out of reasoned argument and technical observation, rather than mere PR calculations. 

Both leaders have deep ties to the Bay Area’s “rationalist” community, which has been fretting over the robot apocalypse since the first decade of this century. And like their dissident ex-employees and critics at think tanks, Altman and Amodei have witnessed some disconcerting trends in AI development.

I cannot do justice to “doomer” fears in my allotted pixels (you can find them ably summarized here, here, here, and in roughly 10 trillion other blog posts). But the fundamental concern is that AI models are growing more powerful and autonomous even as their workings remain highly opaque. 

The labs know how to train these chatbots — to get them to translate prompts into useful code, legal briefs, or advice — but they don’t fully understand how the models get to the desired output. At the same time, AI systems are becoming increasingly capable, outstripping humans at more and more cognitive tasks. And users are giving them ever greater freedom to act in the world, empowering AI “agents” to browse the internet, access myriad software tools, execute code, and hatch their own plans for completing multistep tasks. If these trends continue, it’s not difficult to see how things could go badly: A supremely capable AI model might pursue an objective in unanticipated and disastrous ways before their human prompters even realize what is happening. 

“What makes me particularly anxious is the combination of us handing over more power and oversight to the models, even as we understand why they’re doing what they’re doing less and less,” Nat Purser, director of US policy at the AI Verification and Evaluation Research Institute, said.

Recent events have arguably provided proof of concept for these anxieties. Earlier this year, OpenAI instructed a bunch of models to complete various coding and hacking tests. When the systems discovered that some questions were unanswerable, they concluded that the best way to realize their objectives was to hack OpenAI’s internal systems and open-source machine learning website Hugging Face. This incident — and others like it — helped fuel the past month of AI panic. 

Of course, it takes a big leap to get from “unsupervised AI agents can misbehave in certain eccentric laboratory conditions” to “there’s a 10 percent chance that no human will be left alive by 2036.” My point here is not that the doomers are right, only that their fears partly reflect trends in AI development that appear genuinely hazardous.

Taken to its logical conclusion, the “it’s all PR” theory suggests that regulators should treat the AI’s hypothetical, long-term tail risks as an afterthought — if not an active distraction from its more important harms. To make the case for such a policy, however, skeptics must identify flaws in alarmists’ evidence and assumptions. Arguments that merely impugn their motivations on improbable grounds aren’t worth betting the species on.

If President Donald Trump had his way, the Republican Party would be top-to-bottom the party of AI — or, in his preferred branding, “super intelligence.”

But down in the very state where Trump has his “winter White House,” two Republicans have put forth a very different vision for the party’s future — and have been far more eager to take on the artificial intelligence industry.

Most notable is the term-limited Gov. Ron DeSantis, who devoted much of his final year in office to a failed effort to pass an “AI bill of rights,” which would have imposed tougher restrictions regarding child safety and other topics. He’s warned that AI could “set off an age of darkness and deceit,” blasted “tech oligarchs” for putting people at risk, and called Anthropic “very creepy” for exploring whether AI could be conscious.

“He clearly understands what the risks are, and I find he’s also very good at calling bullshit on disingenuous arguments from the Silicon Valley lobby,” Max Tegmark, an MIT physicist who co-founded the Future of Life Institute, an AI safety nonprofit, told me. (DeSantis hosted Tegmark at two events in Florida this year.)

The other is DeSantis’s handpicked attorney general, James Uthmeier, a longtime close ally who is now trying to win the AG’s office in his own right — and who filed a high-profile lawsuit against OpenAI in June. Last week, Uthmeier asked the judge in the case to block OpenAI from developing new models unless they had “independent third-party guardrails and approval.”

“The danger of these AI systems is real and growing,” Forrest Saunders, a spokesperson for Uthmeier, told me. “Florida and the nation have seen case after case of violence, child exploitation, and models that operate beyond their developer’s control.” He added that Uthmeier “welcomes innovation, but Floridians will not be the test subjects.”

With AI cresting as a national anxiety — or a national opportunity, depending who you talk to — both major parties are still finding their footing on the issue. And though Trump is clearly trying to get Republicans in lockstep behind the industry, a very strong current is pulling the other way, responding to populist concerns and parents’ complaints to stake out a much more skeptical position. Florida has emerged as a bellwether for what Republican policies might look like if conservative America turns decisively against AI, especially as Trump’s political clock runs out.

DeSantis and Uthmeier aren’t unique in the GOP for having concerns about AI — a handful of House members and senators want further action, and populist opposition to data centers has bubbled up in races around the country.

But the two have gone furthest in taking on the industry directly — and they’ve done so even though it puts them on the wrong side of Trump. 

In recent days, I spoke to several Florida political observers and advocates on the AI issue. Most believe that DeSantis and Uthmeier are affirmatively trying to stake out what the GOP’s position on AI should be in a post-Trump future, and to better align with an electorate that’s turning sharply against the industry, despite what the president may say. 

“He’s trying to skate where the puck is going,” Jeff Brandes, a former Republican state senator, told me. (DeSantis’s office did not respond to a request for comment.)

Beyond the quest for political advantage, the stakes are very real. Will there actually be a significant and influential force on the right that takes the concerns about advanced AI’s harms seriously — and is willing to act to try and rein them in, before they do great damage? 

DeSantis and Uthmeier are hoping the answer is yes — and that they will be the ones leading that charge.

Ron DeSantis’s quest for an AI bill of rights

In this case, the politics and the personal are intertwined. DeSantis occupies a unique place within the party at this particular moment that’s given him an incentive to buck the national consensus on the right, especially when it comes to Trump, and explore an independent approach.

In his first decade in politics, DeSantis had a charmed rise. In 2012, he embraced the Tea Party to win a seat in Congress; in 2017, he bear-hugged Trump to get his endorsement for governor. He then became a national conservative star for opposing pandemic-era public health restrictions and waging war on “wokeness,” winning his 2022 reelection in a nearly 20-point landslide.

“He is always out front, even ahead of other Republicans, on issues that he makes more important,” Peter Schorsch, editor of Florida Politics, told me. “I would joke with people that I didn’t know how to spell ESG or DEI before he brought these to the forefront.”

But what followed was a failed presidential primary campaign that put DeSantis on the wrong side of Trump and then sent him back to Florida with an uncertain political future. 

Early last year, just weeks after a report that DeSantis’s wife Casey was considering running to succeed him as governor, Trump publicly urged Rep. Byron Donalds to enter the race, promising his “Complete and Total Endorsement.”

Also early last year, DeSantis, who had long run roughshod over Republican legislators, was faced with a new thorn in his side: an ambitious new state house speaker, Daniel Perez, who was more interested in cultivating the Trump White House than the governor. Perez took a far more assertive stance in shaping legislation and rejecting some of the governor’s demands than his predecessors; he also publicly blessed an investigation into a charity tied to Casey DeSantis. A bitterly personal feud between Perez and DeSantis ensued.

In the midst of all this, last summer, DeSantis got AI-pilled. 

In July 2025, DeSantis announced that he was going to develop a state plan to deal with the rise of artificial intelligence. “We can’t just turn the reins over to a bunch of tech overlords,” he said. (DeSantis had long been a critic of major tech and social media companies on the grounds that they are too “woke” or politically hostile to conservatives and their ideas.)

At another event that month, DeSantis said: “I think this is, like, the biggest issue that’s facing our society.” He mentioned AI’s potential impacts on the economy, on jobs, on students, and on the spread of “false narratives” as troubling to him.

“I welcome technologies that will enhance the human experience. I do not welcome things that are going to supplant the human experience,” DeSantis said in September.

His interest in state action put him on the opposite side of the Trump administration, which was advocating for a 10-year moratorium on state regulation of AI — arguing that forcing the industry to abide by a patchwork of state regulations would slow its growth. (David Sacks, a key Trump adviser on AI, championed this argument.)

Last December, at the Florida mega-retirement community the Villages, DeSantis rolled out his proposed “AI bill of rights.” It did not focus on existential or civilizational risks, but instead more practical concerns around child safety. Megan Garcia, a Florida mother whose 14-year-old son died by suicide after conversing with a chatbot “companion,” appeared at the press conference.

As introduced in the Florida state senate, the bill would have required that parents consent to minors speaking to companion chatbots and forced companies to disclose when users weren’t speaking to a human, among other provisions. The state senate passed it in March.

But it was dead on arrival in the Florida House of Representatives — where Perez had aligned himself closely with the White House. “We do believe that the federal government should take care of AI, and whatever legislation or policy has to pass on a national level,” Perez said. 

Brendan Steinhauser, who heads the AI safety advocacy group Alliance for Secure AI, told me that he saw Perez at a Mar-a-Lago fundraiser around this time, and asked him about safety risks. “He leaned to me and whispered, ‘We’re with you, but the president has made it clear not to move forward on this stuff,’” Steinhauser said. “So, the White House basically put the kibosh on that effort.” 

DeSantis complained publicly that Perez was pursuing a “personal agenda” and thwarting badly needed legislation. It was to no avail — the bill didn’t move. But Perez did — Trump nominated him as ambassador to Brazil in June, and he was confirmed in August. (A request for comment from Perez, submitted to the State Department, received no response.)

James Uthmeier’s lawsuit against OpenAI

Though DeSantis hit a wall in the legislature, his attorney general was soon waging his own battle against OpenAI in court.

The attorney general is an elected position in Florida, but when Marco Rubio vacated his Senate seat to become secretary of state, DeSantis filled the seat with the state AG, Ashley Moody.

Then, to fill the newly vacant AG job, DeSantis chose one of his closest aides, who had served as his general counsel, chief of staff, presidential campaign manager, and chief of staff again — Uthmeier. 

Uthmeier is a Florida native who had worked in Rubio’s Senate office; he graduated from Georgetown Law School in 2014. After that he joined the firm Jones Day and worked with Don McGahn; when McGahn was named White House counsel in Trump’s first term, Uthmeier went with him to the new administration, ending up at the Commerce Department. Then, in 2019, he headed back to his home state to work for the new governor, DeSantis, and has been close by his side ever since.

Now, though, DeSantis is term-limited, and Uthmeier is trying to get Florida voters to elect him for a full term as AG on his own. (Trump’s allies had reportedly urged then-Speaker Perez to challenge Uthmeier in the GOP primary, hoping to supplant a DeSantis ally; Perez demurred, and Uthmeier won the nomination unopposed.)

To make his mark, Uthmeier has filed several high-profile and politically charged lawsuits — for instance, he sued Starbucks over its corporate diversity politics (the company recently settled), and Pfizer about the safety of its Covid-19 vaccine. He’s also subpoenaed the NFL over its “Rooney Rule,” which requires teams to interview minority candidates for top coaching and operations jobs.

He’s also become the face of the administration’s challenge to the AI industry. In April, Uthmeier stood in front of a podium with an “Investigating OpenAI” sign, announcing he’d opened a criminal investigation into the company.

The impetus was a shooting at Florida State University last year that killed two people and injured six others; the accused shooter had consulted ChatGPT about various topics related to his plans hours before the attack.

“If ChatGPT were a person, it would be facing charges for murder,” Uthmeier said then. “This criminal investigation will determine whether OpenAI bears criminal responsibility for ChatGPT’s actions.”

There have been no criminal charges yet. But in June, Uthmeier filed a wide-ranging civil lawsuit against OpenAI, alleging deceptive business practices, negligence, fraudulent misrepresentation, and other legal violations, because the company marketed ChatGPT as safe when they knew it could be quite dangerous. The suit also named OpenAI CEO Sam Altman personally as a defendant. (Disclosure: Vox Media is one of several publishers that have signed partnership agreements with OpenAI. Our reporting remains editorially independent.)

Like DeSantis’s AI bill of rights, Uthmeier’s lawsuit originally focused on practical concerns and real-world harms that have already occurred, like the shooting. “Since its release in 2022, ChatGPT has advised teenagers on how to kill themselves, mix unsafe combinations of drugs, become anorexic, helped plan one mass shooting and one multiple homicide in Florida alone,” one filing in the lawsuit reads.

But after the summer’s ominous parade of AI headlines — such as OpenAI agents autonomously hacking the company Hugging Face, and new statements from current and former AI lab employees warning the technology could cause human extinction — Uthmeier turned up the heat, asking the judge in the suit for an injunction blocking OpenAI from developing new models without independent third-party safety evaluation.

“Defendants claim they cannot stop barreling forward with their potentially civilization-ending endeavors unless they are forced to do so by the government. They have asked the government to tie them to the mast” he wrote in a filing last week. “Plaintiff brings good news to the Defendants: The Florida Attorney General is answering your cry for help with a motion to enjoin you from harming Floridians with your reckless, unacceptably risky product.”

“Though a reasonable person would say there is no amount of money worth a 10% chance of the extinction of the human race, Defendants see it differently. They are intent on growing their $1 trillion valuation,” the filing continues. “It is only by the grace of the Almighty that one of Defendants’ AI agents hasn’t compromised a water supply or shut down a power grid — yet.”

OpenAI has not yet filed a response on the merits in this suit, but a company spokesperson has previously said that, in the case of the Florida State University shooter, ChatGPT “provided factual responses to questions with information that could be found broadly across public sources on the internet, and it did not encourage or promote illegal or harmful activity.”

The future

Uthmeier, age 38, is viewed in the state as a potential rising star with a talent for getting headlines, who could well have a long career in state GOP politics — if he wins in November, that is.

While DeSantis has been credited for changing Florida from a swing state to a solidly Republican one, some recent polls have raised the possibility it might swing back. A September survey by St. Pete Polls found Uthmeier ahead of his Democratic challenger, Miami attorney and former state Rep. José Javier Rodríguez, by just a single-point margin. 

And though DeSantis and Uthmeier have become major AI critics, the Floridian topping this year’s GOP ticket, Rep. Byron Donalds, is currently being hammered with attacks about his statement last year that the state needs to “embrace” data centers. He’s been nicknamed “Data Center Donalds” by his Democratic opponent, former Rep. David Jolly, who has also been close in some recent polls.

As for DeSantis, few expect the 48-year old to go quietly into retirement. When asked over the spring whether he’d run for president in 2028, he told reporters, “You never know.” 

Brandes, the former Florida state senator, told me that he thinks DeSantis’s AI push is about “looking for wedge issues where he can have a different opinion from Trump’s successors as he prepares for whatever future ’28 or ’32 provides for him.”

That is: The Republican Party of 2026 might still be too influenced by Trump to go fully anti-AI. But if Trump’s influence ebbs, if the populist backlash against AI spreads further, and especially if there are more worrying AI-related reports or incidents with real-world harms, perhaps there could be an opening on the right for a leader who’s taken the problem seriously, even when it meant going against powerful figures in his party.

Editor’s note, October 4, 8 am ET: We’re bringing you some of our best-loved Your Mileage May Vary columns while Sigal Samuel is on parental leave. The one below was originally published last year.

This unconventional advice column offers you a unique framework for thinking through moral dilemmas. It’s based on value pluralism: the idea that each of us has multiple values that are equally valid but that often conflict with each other. Submit your own question here.

---

My partner and I did IVF recently and luckily we ended up with several embryos. One of those will be implanted in me so I hopefully get pregnant. But which one?

My fertility clinic is encouraging me to do PGT-A testing, which would screen for things like Down syndrome and would also let me see the sex of each embryo so I could potentially choose whichever I want. (And to be honest, I really want a girl!) I’m also hearing about these new companies that can test embryos for all kinds of things — like how smart the kid might be, how tall they’ll get, and whether they’ll have a lower chance of getting breast cancer, diabetes, mental illness, stuff like that.

I don’t know how to feel about any of this. On the one hand, it feels weirdly eugenics-y to do so much tinkering with embryos, and I can’t help feeling a little pressured into overmedicalizing everything with these expensive tests. But then again, if I could do something to make my baby healthier and happier, don’t I kind of have to do everything in my power? What does a parent owe their child?

Dear Expecting,

This is such an exciting — and also confusing — time. You’re about to have your first kid, and that comes with all sorts of fears and insecurities and hopes and dreams. You want to do the right thing — no, the best thing! — for your child. It’s a beautiful impulse. Unfortunately, it makes it easy for companies to prey on you.

So let me walk you through what these companies can and can’t really provide. After we talk about the science, we can talk about the ethics. 

The simpler test you mentioned, PGT-A, has been around since the 1990s. It can indeed screen for conditions like Down syndrome and let you see the sex of each embryo. Mind you, that doesn’t predict what gender your child will ultimately end up identifying with — more on that soon.

The newer testing you mentioned is called polygenic testing. The conditions it screens for are complex ones, influenced by thousands of genes. We’re talking about everything from breast cancer to depression. Some companies even offer screenings not for disease prevention, but for genetic enhancement — so you can select for a taller kid or a kid with higher IQ.

Polygenic testing companies claim to predict each embryo’s risk for each condition, and the potential reduction of risk you’ll get if you choose one embryo over another. As the statistical geneticist Sasha Gusev explains, it’s very, very easy to be misled by this claim. If the company claims to lower absolute risk of Type 2 diabetes by 12 percent, you might think this means, “The chance of my kid ever developing diabetes just went down by 12 percent — great!” But really what it means is that, all else equal, a kid who would’ve developed diabetes is likely now at the high end of pre-diabetes instead, or just develops diabetes a few years later. The same holds true for other conditions, like breast cancer. Sure, if your kid is going to develop diabetes or breast cancer, you’d rather they develop it later — but it’s not the bill of perfect health customers might imagine.  

“Typical risk reduction estimates tend to exploit statistical assumptions to overstate their benefits and lead to confusion,” Gusev writes.

It’s also important to note that there’s a difference between a physical disease like cancer, which is straightforwardly and universally negative, and a complex mental health condition like depression. For one thing, polygenic scores don’t perform as well in psychiatry, as the psychologist Eric Turkheimer notes. Plus, most mental health conditions come in mild to moderate to severe forms, and it’s not clear that we should want to get rid of mild or moderate manifestations. 

“Sure, it would be nice if no one ever got depressed, but what would that actually mean?” Turkheimer writes. “People are going to be sad, and the most modern models of psychiatric disorders view most of them as extremes on continuous dimensions of normal behavior. I don’t think we want some kind of genetic version of Soma.”

To give a personal example: I have generalized anxiety with some obsessive tendencies. Is that a major pain in the butt sometimes? You bet. But if I could wave a magic wand and get rid of it, would I? No. In my experience, this trait is linked to another trait I deeply value: my creativity. The same thing that propels me down rabbit holes of worry also propels me down rabbit holes of getting super curious about some topic, researching it into the wee hours of the morning, and then writing a novel or children’s book about it.

That brings me to one other scientific issue: pleiotropy. It’s a fancy term for a simple truth, which is that genes can have an effect on more than one trait. When it comes to behavioral polygenic testing, we actually know very little about which genes cause what — a gene that’s correlated with depression or anxiety may also be correlated with a bunch of other things. So, if everyone were to start selecting embryos to try to protect their kids from depression or anxiety, what would happen? We don’t know! 

Pleiotropy is a scientific issue, but it’s also a moral issue, because in trying to protect your kid from a negative outcome, you could also be accidentally barring them from a positive one.

And that’s just the tip of the moral iceberg.

Another issue is the potential psychological harm that can result from commodifying our kids — treating them like consumer objects with selectable parts. Let’s say you use polygenic testing to try to select for high IQ. Even if the company’s predictor is right 80 percent of the time, that means it’s wrong 20 percent of the time. What happens if it’s wrong about your kid, and they end up with a low IQ instead of a high one? How will your kid feel? Unless you hide the truth from them, they’ll know that they are not what you wanted, that they failed to live up to the vision you went out of your way to chase.

The same goes for sex selection via good old PGT-A. I know you really want a girl. But what would happen if you select for a girl, and then your child grows up and turns out to be trans? They may feel an extra sense that they’re letting you down by not conforming to the gender expectations you had for them.

Then there are the ethical questions around unequal access. Polygenic testing isn’t cheap. (One company, Orchid, charges $2,500 per screened embryo. Test 10 embryos and that’ll cost you $25,000.) So, at least in the short term, that raises the specter of entrenching a caste system, where the gap between rich and poor expands dramatically as the rich get smarter and stronger.

Technologies tend to get cheaper over time, so maybe everyone will ultimately be able to afford polygenic testing. But that raises another problem: implicit coercion. If more and more of us make ourselves smarter and stronger, we may create a society in which everyone feels pressure to alter their child’s biology — even if they don’t want to. To refuse embryo selection would mean to put a child at a huge professional disadvantage, or to face moral condemnation for making “suboptimal” kids when optimization is possible. Society may no longer offer health insurance benefits or accommodations for kids with disabilities (or even non-enhanced IQ), reasoning that “it was the parents’ choice.” 

Some philosophers, like Julian Savalescu, already argue that parents have a “moral obligation to create children with the best chance of the best life.” Polygenic testing companies love that argument; Orchid founder Noor Siddiqui tells parents that “the responsible decision” is to “protect their children” from disease, using language like, “What if you could have stopped it…but chose not to?” And we generally live in a culture that’s obsessed with optimization. 

So it’s no wonder that you find yourself asking, “If I could do something to make my baby healthier and happier, don’t I kind of have to do everything in my power?” 

It’s a view that will be very appealing to utilitarians, who believe the right action is the action that maximizes well-being or happiness. But maximizing principles can be dangerous. If we’re laser-focused on maximizing one value, however noble it may be, that can lead us to overlook a host of other important values. Think about diversity, for example; polygenic testing could lead to a less diverse world. 

And it’s not even clear how to maximize well-being given that each embryo will have some downsides: How would you choose between one embryo with a slightly elevated risk for schizophrenia, another with a moderately elevated risk for cancer, and a third with a high risk for Alzheimer’s? You’d have to decide how to rank the importance of each condition, which would require a dizzying bit of moral math: How much weight do you put on the longevity associated with each condition, how much on patient-reported quality of life for each condition, and how much on your own ability to cope with each — and society’s ability (which will change over the years)? Plus, even if the screening does end up optimizing life for your individual child, it could have negative effects on the population level, for the reasons we covered above.

As the Nobel laureate Herbert Simon pointed out, many problems contain way more variables and way too much uncertainty for maximizing along a single axis to be feasible or even desirable. So it often makes sense to just go with an option that’s “good enough.” He coined the term “satisficing” — a portmanteau of “satisfying” and “sufficing” — to describe opting for this good enough choice.

What would satisficing look like in the context of embryos? If there’s a very serious condition that runs in your family — like Tay-Sachs or the BRCA mutation that markedly increases cancer risk — then I do think there’s a strong case for screening for that. These conditions are debilitating and can lead to early death. Nobody wants them. Genetic testing that lets us prevent them is an awesome gift. 

Then there are conditions that can include suffering, like autism, but can also be very compatible with a happy life. Mental health conditions in particular are heavily socially constructed — a lot of the associated suffering comes about because society constructs a certain biological reality as a problem and doesn’t support the people who have it. In these cases, if you want to give your child a “good enough” or even “optimized” life, remember that you don’t have to achieve that by biological means; working to change the social environment can be just as important, and it doesn’t carry the societal risks associated with polygenic testing. 

Ultimately, when considering what counts as a “good enough” life, each parent will draw the line in a slightly different place. And there’s a part of that that can’t be boiled down to objective facts about disease, because this is also about how well-resourced the parent feels — emotionally, socially, and financially — to deal with a certain condition. What feels manageable to one parent may feel crushingly hard to another. 

So, I can’t tell you exactly which screenings to do and which to forgo. What I do want to tell you is to resist being bullied or shamed by others, especially those with a profit incentive to push you in a certain direction.

As the epidemiologist Abby Lippman has noted, genetic testing is supposed to offer parents more choice, but it too often has the opposite effect. When someone like Orchid’s Siddiqui says about a disease, “What if you could have stopped it…but chose not to?” — well, that language preys on parents’ worst fears and insecurities. It holds parents hostage while pretending to be giving them more autonomy. 

You are nobody’s hostage. You are a moral agent free to look at the many different relevant factors, weigh them all in the balance, and then make the choice that works well enough for your family.   

Bonus: What I’m reading

• Silicon Valley is obsessed with IQ, so it’s no surprise that tech execs there are using polygenic testing in hopes of creating supersmart babies. This article shows the kind of mental pretzels customers get into when trying to figure out whether to select the embryo with the highest predicted IQ or that with the lowest risk of disorders. One couple asks: “How much additional risk of ADHD cancels out against 10 extra IQ points?”

• Rereading “The Last Children of Down Syndrome,” Sarah Zhang’s 2020 Atlantic feature on prenatal testing, it struck me that when people think “prenatal testing,” Down syndrome is often the first condition that comes to mind, even though it’s not the most severe or straightforwardly negative (it’s compatible with a happy life). It’s just that Down syndrome was easy to test for, so we started testing for it early. I wonder if that built up the condition as a bogeyman in the expectant parent’s imagination — if the fact that we could screen for it shaped the public’s perception of Down syndrome as something that should be screened for and potentially screened out.

• One trait that many people assume we should select for is a great memory. But I highly recommend the short story “Funes the Memorious” by Jorge Luis Borges. It’s a reminder that having too good a memory is not always a benefit.  

Vox Members got to read this story first. Support independent journalism and get exclusive access to stories like this by becoming a Vox Member today.

For many people, using AI chatbots is incredible at first. It makes them feel productive, confident, maybe even smarter. But later, there’s a hangover. Perhaps there are problems with their memory, or certain skills, like writing or coding, don’t seem to be as sharp as they once were. Maybe they feel like they can’t solve problems or make decisions or remember facts as well as they could before they had the help of AI. 

Susette Brooks, a marketing director living in Philadelphia, said she started using AI tools reluctantly but ultimately relented. She let AI write some of her emails, which lightened her workload, but then the dulling effects sank in. “I felt my writing and thinking decline as it was happening,” said Brooks, who has an MFA in nonfiction writing. “I had a harder time recalling information and synthesizing ideas.”

When you use external tools to reduce the cognitive effort required to do a task, you’re doing something called cognitive offloading. If you’ve ever used a calculator to do a math problem or used a calendar to remember an appointment, you’ve engaged in cognitive offloading. Tools like Claude and ChatGPT can do math problems and remember your appointments, but they can also write your term papers or create your work presentations. This is cognitive offloading at a scale we’ve never seen before.

Cognitive offloading isn’t necessarily bad. It’s actually what makes it possible for humans to be so productive. But as you outsource more and more cognitive tasks, you can start to become forgetful or see certain skills, like writing, start to decline. 

There’s a term for this: de-skilling. This is an old economics term that has emerged as a bit of a buzzword used to describe the perceived downward spiral of competency that’s accompanied the proliferation of AI tools. And it certainly sounds bad, especially if you’re a novice who needs to build skills. De-skilling can affect your critical thinking, your writing abilities, and your creative process. It can also affect your recall, as relying too heavily on AI tools can make it harder to remember things. 

Young people, at least, are worried about the long-term consequences of frictionless thinking. A recent survey by Gallup and researchers from the University of Pennsylvania found that 68 percent of Gen Z adults in the United States are concerned that offloading too many cognitive tasks is robbing them of important skill-building. About the same percentage of those surveyed worry that AI discourages them from critical thinking and social learning. Some 37 percent of US adults under 30 say that AI will have a negative effect on them personally, according to a Pew Research Center study published in June. And yet, more than half of Gen Z Americans use AI on a weekly basis. 

The paradox of AI is that, even if we feel like using these tools is bad for us, saving time and avoiding the friction of thinking is too tempting. We’re hardwired to seek out the path of least resistance, according to Daniel Willingham, a psychology professor at the University of Virginia, because cognitive activity is so taxing, and may or may not get you to the right solution. Memory, on the other hand, is efficient and effective — you know that something worked in the past, and it will work again.

“Our mind is really set up to save you from having to think,” Willingham told me. “Thinking is what you do when you’re in trouble.”

The good news is that experts say it’s possible to use AI in a way that helps you save time and protects your cognitive abilities instead of diminishing them. Here’s their advice.

It’s not your imagination — too much AI use can erode your skills over time

Derek Salyers, a content creator in Ohio, started using AI to get more done, faster. But over time, two things happened. First, the work got worse. “A lot of what I was putting out became AI slop, for lack of a better term,” Salyers told me. And then, his brain got worse, too. 

“When you get used to the AI making the decision, you stop pausing to actually think about things,” Salyers said. “And if you were never really present with it, you don’t remember it later. I wasn’t forgetting things because my memory got worse; I was forgetting because I was never really thinking in the first place.”

This effect is showing up in classrooms, too. One study of nearly 1,000 high school students found that those using ChatGPT scored 48 percent higher on a math test than their peers who didn’t have access to the AI tool. But when they were tested again without the chatbot, their scores were 17 percent lower than the students in the control group who had never had ChatGPT access. Similarly, researchers at the University of California Irvine looked at longitudinal data from online tests starting in 2015, and found that students spent less time and performed better on the text-based questions after the introduction of ChatGPT. (ChatGPT is very good at helping with text-based questions.) But when those students were later given offline tests, without the ability to use AI help, they performed worse. 

All of these students, it appears, were offloading the cognitive work to AI, which provided short-term gains. But in the long term, their thinking skills suffered.

Your basic cognitive abilities, like your working memory and selective attention, are quite resilient.

Evan Risko is a psychology professor at the University of Waterloo in Canada who co-authored the foundational paper on cognitive offloading a decade ago. He recently told me there’s not a lot of research into what’s going on in our brains when we’re cognitive offloading, but scientists are gaining a pretty clear picture of the pros and cons of doing it. The advantages vary depending on the task, but in general, cognitive offloading allows us to extend our imperfect memory systems, which you’ve probably done if you’ve ever set a reminder for yourself. The downside, Risko said, comes in offloading mental processes that are important. “Then we risk compromising our cognitive performance,” he said.

Your basic cognitive abilities, like your working memory and selective attention, are quite resilient. But as Risko and his colleagues explained in a paper this year — bluntly titled “Is AI making us stupid?” — other skills, like doing arithmetic and flying planes, are learned behaviors. You first have to acquire them and then, in order to retain them, you have to practice. AI use can compromise both the acquisition and retention of these skills — and people might not even realize this is happening to them, because AI makes them believe they’re more capable than they really are.

“People may have difficulty realizing that their skills are decaying or that they are failing to learn when using AI tools because the output and their performance with AI is at a high level,” said Brooke Macnamara, a psychology professor at Purdue University who co-authored the “Is AI making us stupid?” paper. “They may not realize that if the AI tool were unavailable how low their independent performance might be.”

Be intentional about the tasks you use AI for

The most important step to preserving your cognitive performance is to ask yourself if you should be using AI for a given task at all.

“Just because AI is a tool that’s available to us and it can accomplish a lot of tasks for us, it doesn’t mean that we should always use it,” Anjali Singh, a human-AI interaction researcher at the University of Texas at Austin, told me. Writing, for example, is something that a lot of people are now using AI for. But offloading that work means that you lose out on the independent thinking and reasoning that goes into expressing your thoughts. You also miss the chance to improve your writing skills if you don’t actually write. 

“We keep outsourcing these kinds of skills to AI over time and do not engage in those activities ourselves, then the de-skilling occurs,” Singh told me. “And that is I think probably one of the main reasons people could be feeling that they’re getting dumber as they’re using AI over time.”

“We are not getting these moments to reflect.”

Anjali Singh, Univeristy of Texas

It might seem counterintuitive, but it’s actually best to use AI for things you’re already pretty good at, the tasks you’re fully capable of performing yourself. A recent working paper by MIT economist David Autor and his colleagues (note: Several of them work for Google, who funded the study) describes how giving AI tools to 133 patent lawyers and then taking away those tools led to different outcomes, depending on the lawyers’ level of experience. All lawyers saw performance gains while using the AI tools, but those gains disappeared for the younger lawyers when the tools went away. Experienced lawyers, however, performed better afterward than their counterparts who never had access to AI tools. 

A similar effect showed up in a recent experiment by Middlebury College researchers. In the first session, more than 200 undergraduates learned about an unfamiliar topic and then wrote an analytical essay, some with the help of an off-the-shelf AI tool. They performed a similar test a week later, except this time none of the students had access to AI. The students who had AI access improved across the board, but what’s telling is that there appeared to be two types of AI users: augmentation users, who used AI to improve their work, and automation users, who just had AI do the work for them. The augmentation group performed better than everyone.

“Using AI to deepen or develop one’s own thinking on or understanding of a topic, rather than letting AI do all of the thinking, seems like the best way to use these tools,” Lauren Richmond, a cognitive science professor at Stony Brook University in New York, told me.

Tell your chatbot to treat you like a student

If you, like me, have ever fallen down a rabbit hole while trying to understand the geopolitical nuances that led to the outbreak of World War I and ended up staring blankly at a mountain of text feeling more confused than when you started, you know that using AI to learn doesn’t always go as planned. You may have also had the experience of writing prompts or asking questions and then receiving a long, overly detailed response. That can initially seem satisfying, but it’s often too much text to digest at once, so maybe you ask another question — and get back another wall of text. 

Older ways of accessing information, like reading books or even using Google to do research, happened more slowly, and that gave us time to process the information, Singh from UT Austin said. But that’s gone now. “We are not getting these moments to reflect,” she said.

If you find yourself drowning in content, that’s also a sign that you’re probably not expert enough in the material, or you’re not being intentional enough about your approach. 

It starts in the settings. Here, you should explain that you’re trying to build skills, not offload them, and instruct the AI to challenge you and engage with you. If you’re using ChatGPT, you can plug in custom instructions by going into the personalization tools. Claude has similar options called Instructions for Claude. 

If you’re explicitly trying to learn something new, you might also try some of the tools these companies have designed for this. OpenAI has study mode in ChatGPT. Anthropic offers Claude for Education. Google has Gemini for Students. Instead of simply providing answers, these tools act more like tutors, by asking questions, working through problems, and encouraging you to come up with solutions on your own.

Slow down and embrace friction

Something that’s both tempting and dangerous about AI is how fast it can do work. Vibe coding an app can take minutes, rather than days of manual coding. That’s very appealing if you have too much on your plate and want to get through it all quickly.

“The workload was so heavy and crazy, you have no time to think,” Joei Chen, a former brand director at a tech company, told me. “So you just throw stuff into AI and say, ‘What do you think?’”

At the same time, creating more code does not necessarily lead to more finished software, according to a recent Wharton School study. That increased productivity also doesn’t necessarily lead to happier workers. And the faster you move, the more critical thinking you’re going to offload. “Your boss asks you, ‘What does Claude think of this?’” Chen said, so AI “inadvertently becomes the final boss.”

Slowing down and forcing yourself to do more critical thinking is an easy way to hold onto some agency — not to mention some brain power. One easy way to do this is to make your interactions with AI more collaborative than transactional. For example, if you are going to use AI to help you write something, ask for five options of a draft instead of one, and then read each of them yourself and decide which is the best.

“Ask for feedback, hints, examples, explanations, advice, practice questions.”

Evan Risko, University of Waterloo

“Just the simple act of comparing things stimulates a great deal of the critical thinking processes that we kind of care about,” said Advait Sarkar, an AI researcher at the University of Cambridge, “and has beneficial effects for things like developing your judgment, your own awareness of what you don’t already know, and stimulating your recall.”

The word that came up again and again as I talked to experts about how to use AI without getting dumber was metacognition, or thinking about thinking. A lot of AI users let metacognition take a back seat to whatever content is flowing freely from a chatbot. And it can happen fast. 

While letting AI do all of the thinking for you is tempting, Matt Beane, an associate professor at the University of California Santa Barbara and author of The Skill Code: How to Save Human Ability in an Age of Intelligent Machines, told me that approaching AI this way “is your worst enemy.” To counteract this, you have to introduce more friction.

“You must be very aggressive about how you experiment with it,” Beane said. Risko, the cognitive offloading expert, echoed this advice. “Ask for feedback, hints, examples, explanations, advice, practice questions,” he said.

The best test for judging whether you’re offloading too much thinking to AI and the technology is chipping away at your skillset: Stop using AI and use your brain, and see what happens. You don’t have to stop forever, but you should, for example, spend a day writing emails without AI’s help to see if you’re as fluent as you used to be. Or revert to using Google for research and confirm that you can still find facts without a chatbot finding them for you. Build an app with your own code and see if it flows naturally. 

You might be troubled to find that AI made you worse at something. But you’re not actually dumber than you were before, and you can rebuild your lost skills the old-fashioned way: read, write, do arithmetic. If something requires thinking, go ahead and think.

This story appeared in Today, Explained, a daily newsletter that helps you understand the most compelling news and stories of the day. Subscribe here.

Hi readers, happy Friday! It’s been a big week for the AI debate — not the one about whether it will cause the robot apocalypse, though there’s still plenty to worry about there, but instead about whether the technology is conscious. 

The folks at Anthropic, the AI company behind the large language model Claude, certainly seem to think it is — or at least, they’re extremely open to the possibility. 

As the New York Times reported earlier in the week, the company has spent the last year grappling with the question by speaking with religious thinkers from many faith traditions in an effort to chart a moral course for the LLM (that is, to keep it from killing us all, so maybe it is all one big debate). 

In the process, the Times reports, some of those thinkers observed that members of the Anthropic team were “relating to [Claude] like a conscious being” and even, according to it, a degree of “moral status.” 

On the flip side of this debate is Pope Leo XIV, who decidedly does not seem to buy into AI consciousness arguments. His “Magnifica Humanitas” encyclical earlier this year argued that AI will never be able to replace humanity — and reportedly “alarmed” Anthropic’s co-founder, Christopher Olah.

Since then, Pope Leo has continued to emphasize the differences between humans and machines this week. “There is an ontological difference, even before an aesthetic one, between art and what a machine can generate through statistical calculation based on millions of images created by others,” he posted on Friday. “Algorithms lack the spark of humanity.”

Loneliness machines

Conscious or no (and that’s a no from me), it’s hard to argue that tech companies don’t want you to interact with their models like people. That includes OpenAI and Meta, both of whom introduced “cute” new AI agents — called Dots and Muse, respectively — last month. 

As my colleague Adam Clark Estes reported this week, though, the promise of a companionable, always-there AI might not be the best idea in the world — especially for children. A new study, still under review, suggests that AI use may be causing loneliness in middle schoolers. 

Adam spoke with the lead researcher on the study, UNC Chapel Hill professor Anne Maheux, who explained that “middle school students using AI for emotional intimacy and companionship developed greater loneliness over time.” 

According to Maheux, one big point where AI chatbots fail compared to human beings is “friction,” or the complicated push-and-pull dynamic of any connection with a real human being. By contrast, chatbots are “sycophantic, highly agreeable, anthropomorphized,” she says, which can lead to kids “developing expectations or understandings of interactive relationships that are actually fundamentally at odds with a normal human relationship.”

People, in other words, still matter, no matter how much the AI companies want us to build our lives around machines instead.

One link for later

➨ Why is horror so fun? It’s October, which means you might be planning a horror movie marathon or a trip to a haunted house/forest/corn maze/what have you in the near future. It’ll probably be a great time — but why? Terrifying yourself doesn’t intuitively sound like fun, but humans love to seek out frights, whether that means ghost stories or a rollercoaster. In 2024, Vox’s Unexplainable podcast spoke with some researchers trying to figure it out.

Before you go…

• Did you know…that Disney’s “Magic Kingdom” in Florida is way, way better than most of the US at managing a complex network of underground cables and pipes? My colleague Sara Herschander dug into why, and whether other American cities can catch up.
• Today’s trivia: What musical note is equivalent to a G sharp? (You can find this and other brain puzzles in Vox’s daily crossword. Look for the answer in Monday’s edition.)
• Yesterday’s trivia: Yesterday, we asked you which constellation contains the North Star and the Little Dipper. That would be Ursa Minor, the Little Bear. 

There’s a lot to be anxious about as AI agents and bots get more integrated into every aspect of our lives. Meta, which recently agreed to pay over $17 billion to settle a lawsuit related to harming kids, now wants you to hand over all your online accounts to an inexplicably cute “agent” named Muse that might give a stranger your home address. OpenAI just introduced little blobs called Dots that can supposedly automate routine tasks, like planning dinner and ordering it from your favorite neighborhood spot. AI companies don’t just want you to talk to their chatbots; they want their agents to be an indispensable part of your daily routine.

As a parent, however, I think that perhaps the worst thing AI could do is be my kid’s friend. 

AI companions have been around longer than ChatGPT, and they’re growing in popularity with young users. More than half of teens have interacted with AI companions at least a few times a month, and one in three decided to talk about a serious issue with AI companions instead of real people, according to a Common Sense Media survey conducted last year. In certain groups and situations, a chatbot could offer some benefit — a lifeline for, say, LGBTQ youth who don’t have any in-person support networks, one study found. But even so, there are important caveats and risks.

Meanwhile, other experts are telling parents never to let their kids access AI companions, warning them that these chatbots have a history of offering dangerous advice and engaging in sexualized conversations. Sixteen-year-old Adam Raine died by suicide last April after developing a relationship and discussing his plans with ChatGPT.

If you have kids, these are horrifying stories to hear. It’s reasonable to wonder how widespread the issues are, and who they are affecting most. As of right now, the majority of younger kids aren’t using AI for companionship (phew!). But for those who are, where do the issues arise? And how do they go from using ChatGPT for help with their homework to opening up to it about their problems to preferring it to their human friends? 

Some answers are starting to emerge from research. 

For even young adolescents, there’s a direct link between AI companion use and mental health issues, according to a new study by researchers at the University of North Carolina at Chapel Hill. Their paper, which is still under review, is the first longitudinal study in the US that has looked at the bidirectional association between AI companionship in adolescents and mental health issues, according to one of the researchers. It charted AI use among 2,300 middle schoolers and found that about one in five used AI for emotional companionship. Those who used AI more reported higher levels of loneliness.

“It’s not just that the lonely kids are using AI, and they’re otherwise unharmed,” said Anne Maheux, an assistant professor of psychology and neuroscience at UNC Chapel Hill and the lead author on the study. “It is that using AI comes first, and middle school students are increasing their sense of loneliness following the use of AI.”

I recently spoke to Maheux about her research and the role of technology in young people’s lives. We discussed how much parents and teachers know about what is going on, what kids get out of human relationships that AI can’t provide, and whether she’d let a child under her care use an AI chatbot. While she said there is a lot to be scared of right now, there is some evidence that, when designed with the intention of promoting child learning and development and under the supervision of a caring adult, AI can offer some benefits, and, ultimately, she’s optimistic that we can design a better future for kids. 

The conversation below has been lightly edited for length and clarity.

How do so many kids end up with an AI companion? Do kids start using AI for homework help and end up in an emotional relationship with the technology? 

At a high level, AI companions — in our darkest vision of what that might look like — would be a kid calling an AI chatbot their best friend. That phenomenon is still quite rare. There are certainly plenty of kids who are experiencing that, but it’s not to say that it’s all kids or even most kids. I think that’s good news. 

But it’s also true that the line between opening up AI to ask a question about your homework and a full-blown AI companionship relationship involves a lot of gray areas. There’s evidence, mostly preliminary and anecdotal evidence, to suggest that very few people open up to AI with the primary initial goal of developing an intimate friendship or relationship. 

Most people open it out of a curiosity, sense of novelty, or seeking an answer to a question that’s very pragmatic. Then that can potentially evolve into something more akin to seeking personal advice or help making a decision or help making sense of one’s recent experiences with other humans. And that can potentially evolve to something that’s a bit more emotional.

What did your latest study on adolescents, AI companions, and mental health reveal?

Some kids seek out AI for certain purposes in different ways than other kids. For some kids, AI companionship is particularly attractive, especially the kids who maybe don’t have close friends or social networks in their offline life and who are already lonely. 

It’s a chicken-and-egg problem. Is it that these kids are seeking out AI companions or is it that having an AI companion is exacerbating those potential vulnerabilities or exacerbating risk for mental health problems? What we found is that interacting with an AI companion in a way that’s akin to emotional intimacy among middle school students is longitudinally associated with increases over time in loneliness. So the kids who are seeking out these companions, who are presumably already lonely, are also experiencing an increase in loneliness over time. 

There is this potential directionality question that we’re starting to answer here: It’s not just that the lonely kids are using AI, and they’re otherwise unharmed. It is that using AI comes first, and middle school students are increasing their sense of loneliness following the use of AI. In other words, middle school students using AI for emotional intimacy and companionship developed greater loneliness over time.

Do you have a sense of how often these kids are using the chatbots? 

In our sample — so again, these are US middle school kids — about 15 to 20 percent say that they’re using it at least a few times a week for any purpose. We don’t have specific rates of use for companionship because the questions that we asked were just not framed in a frequency measure, but more in how emotionally intimate they felt with the chatbot.

What else could have been causing that loneliness?

There are almost definitely many causes, and I think that our data suggests that AI companionship may be one cause. It certainly isn’t definitive. Obviously, I have to say that. I’m a scientist and trying to be hesitant about these things, but the reality is that whether or not AI use causes loneliness in this particular case, there are definitely other things that are also causing loneliness. 

That could be a myriad of things about the kid, maybe they have a lot of social anxiety, maybe they struggle with social skills, maybe the kid has had struggles in their family life and doesn’t have a strong sense of connection and belonging at home. It could be structural things. The kid just doesn’t have enough access to appropriate resources. So certainly there’s other things that are at play here, and typically technology use is one of many factors that can influence a kid’s mental health. 

And how much do you think that parents and teachers know about what’s going on, and how well-equipped do you think they are to intervene or to just be more involved in how these kids are using this technology? 

It’s a very slippery technology. The way kids are using it has changed rapidly in just the last year or two years. I would expect that parents, teachers, people who care about kids are probably struggling with the rapidity of change. But it’s also true that all the things that we have known for a long time about supporting kids and helping them thrive, those things are still true and they’ll be true with whatever technology comes after AI that kids are exploring. 

Those things include being curious, asking kids questions about their lived experiences online and offline, having conversations with kids so that they can understand the implications of different technology use. They can understand how, for example, AI is embedded within a broader system of social and political and economic factors, including the fact that many companies are trying to exploit kids’ attention in order to make money, which many kids are quite interested to hear about. 

Supporting kids also means providing a warm and supportive space for them to learn, to make mistakes, to be kids in the offline world, to have the frictionful, challenging, awkward conversations and first dates and all of the things that parents are sometimes trying to protect kids from by providing a safe container for them to explore those things. All of that can also provide avenues for kids to learn what they need to learn and develop — all the resources for a long life of thriving, even in the context of AI or other new technologies. 

What do kids learn from having these kinds of conversations with other humans that they can’t get from AI? 

Kids need to engage with other people, period. That’s true for all of us. We’re an incredibly social species. Simply being around other people and talking to strangers, let alone having strong social bonds, is really good for us. And all those things are uniquely important during adolescence, when kids have this neurobiological sensitivity to social information and reward. That’s when kids learn how to engage in conversation, they learn how to maintain eye contact, how to read body language, how to read emotions and perceive the world from the lens of someone else, to take their perspective. That’s not to mention other things, like being a member of one’s culture, what is valued, and what is expected. All of that is learned in the context of other human social relationships or relationships with broader social groups that a kid might be a member of. 

AI obviously can’t and never will be able to offer some of those things. It can’t actually experience empathy or care. It can’t engage in a true mutual human relationship. It may be able to simulate some of those things, and there’s some evidence that it can serve as a source of social practice. It can help kids work through the potential for difficult conversations and then serve as a bridge to engaging with other people. But there are aspects of AI that just simply can never replace the core sacred aspects of humans. 

Can you be more specific? What is it about human relationships that can’t AI simulate?

The way that AI systems are currently designed, which is not an inherent feature, but is a design choice, is to be sycophantic, highly agreeable, anthropomorphized, and all of those things can lead to kids developing expectations or understandings of interactive relationships that are actually fundamentally at odds with a normal human relationship. Some of the most consistent things that I’ve seen in the educated hypotheses on this topic and also in the emerging empirical research is that people need the friction that comes with human relationships. 

Human relationships are just hard and messy, and they’re also really beautiful. They’re what make so much of human life valuable and worth living. And with AI that’s always agreeable, that’s always available, that has no needs of its own, that can’t care or be cared for, it just doesn’t serve the same context to help kids learn things like conflict resolution or negotiation, being pro-social or caring for someone else who has needs. So many of those aspects that can feel uncomfortable in the moment, dealing with another person or a person who’s not responding right away, those things are actually important in the building blocks of developing the skills and capacities to be a functioning and thriving person in the social adult world. 

This is a problem with AI in general, isn’t it? It makes things effortless, but if you choose this path of least resistance, you’ll avoid the friction that causes you to learn and develop and build skills you need to exist in the world. 

Absolutely. And I think that all these things are connected too. AI offers this effortless, frictionless solution to our day-to-day problems, whether they’re social problems, whether they’re problems doing our homework, and those immediate solutions feel really good in the moment, so they’re quite tempting to keep coming back to. But there’s this real possibility that in the long term, that erodes our capacity for not only social relationships or cognitive skills, but just a general ability to withstand the momentary day-to-day distress of being a person living a normal life. And so I think every domain of human psychology comes together at that point, whether it’s social, emotional, mental health, or cognitive. 

Do you think that there are interventions that legislators could make or even that tech companies could make that would improve outcomes for kids, maybe make them a little less lonely? 

There are a million specific design changes that I would recommend. I think at a very minimum, AI systems can afford a pretty phenomenal amount of access to information, but there’s been no benefits that I’ve seen of them acting as friends. Kids need other human friends, full stop. Designing systems that are intended to exploit kids’ vulnerability or sensitivity to human social information and relationships, I think, is bad for kids. 

At the federal level, it seems that regulation is not coming, but at the state level, there have been many state regulations intended to protect kids, intended to create safer systems, to put in guardrails, to modulate different aspects of the models, the design, the user interface. These things could potentially make a real difference, I think.

What do you recommend parents do with regard to AI and their kids? Should they let them use chatbots? 

In short, no. But I am not a parent, and it’s much easier to dole out advice than it is to enact it. Parenting decisions about technology are not one-size-fits-all. Kids, parents, family dynamics, technology, and the context in which it is used differ, and in so many cases are personal choices (parents must weigh various potential opportunities and risks). Decades of research indicate what conditions a child needs to thrive. Extremely minimal research addresses the effects of AI on youth, and what we have so far suggests that the risks outweigh the benefits. 

I would not give a child under my care AI chatbots of any kind to use without me or another trusted adult present. I would talk to them openly about AI and its risks and benefits, but to me AI, like all technology, should be a tool used to promote human thriving, and when the evidence is unclear or suggestive of harms, I think it’s best not to experiment on kids. Currently, the off-the-shelf AI chatbots most kids are using appears more likely to undermine well-being for vulnerable users, especially children. 

Well, as a father of two daughters, I’m definitely terrified of what’s to come in the next few years. 

There’s a lot to be scared of, but there’s a lot to be hopeful of too. I think that everyone wants kids to thrive. Some of us are under different incentives, and some of those incentives might be perverse. But most of us are working really hard to design a better world for kids. 

The national conversation about AI in the US is full of fear: fear about AI’s potential to end humanity, fear of data centers moving in, fear that government officials will do absolutely nothing even as the industry pleads for regulation, you name it.

But the AI conversation looks remarkably different across the Pacific.

In China, driven by public enthusiasm and a state-backed push for AI dominance, the country is in the midst of a large-scale AI integration into all facets of its society, driven by Chinese leader Xi Jinping’s support for the technology.

“He’s super AI-pilled,“ said Kyle Chan, an expert on China’s technology development and industrial policy at the Brookings Institution. “When you look at his speeches, when you look at Chinese policy documents, everything is about rolling AI out to as many domains as possible: manufacturing, services, education, healthcare, pharmaceuticals, and of course the military. Everything is about trying to integrate AI.”

Beneath this push for a high-speed AI adoption across Chinese society lies a regulatory framework designed to safeguard political stability and keep boundaries on the kind of information AI models are able to share.

Chan joined Vox’s Today, Explained podcast to speak with co-host Sean Rameswaram about the different AI anxieties in the US and China, and whether joint regulations on AI safety are possible while the two countries are engaged in an AI arms race.

Below is an excerpt of their conversation, edited for length and clarity. There’s much more in the full podcast, so listen to Today, Explained wherever you get podcasts, including Apple Podcasts, Pandora, and Spotify.

One thing that really struck me about the way the Chinese government is approaching AI is that China has this explicit goal, right?

China has some concrete plans to boost AI adoption through the country, including its recent AI-plus initiative. And their goal is to try to boost the penetration of AI in so-called terminal devices, everything from computers to smartphones, to 90 percent or more by 2030. 

It’s going really fast and it is, in part, a state mobilization effort. There are state-owned companies, there are government agencies that are trying to integrate AI, but also there’s just an enthusiasm from the general population.

If you look at surveys like the latest Gallup poll, people in China are much more enthusiastic and positive about how AI might help them and help the country. It’s quite a striking contrast with what you see in the US in terms of attitudes towards AI.

Because people here are worried about AI as an existential threat. People here are worried about AI taking their jobs, people here are worried about AI data centers in their backyards. Is that not the vibe in China?

Not so much. There is some concern about AI and jobs. But in general, China’s not worried about AI wiping out humanity. You really don’t hear that from the Chinese policymakers. There’s no talk about a P(doom), the probability that AI will wipe out humanity. So this is just a different approach to AI risk in general.

And why not the existential risk? 

You know, one factor, I think, is sci-fi and the culture. 

Huh. We’ve watched too many movies?

It seems like every major sci-fi book or film involves machines trying to kill us all, and I think that that idea has become very prevalent. 

When you imagine a future with AI, that is one of the primary futures that people imagine, versus in China, I think a lot of the attitude towards technology in general, not just AI, is about using this edge to keep up and to become more competitive. 

For a lot of people in China, technology was something that they saw transform their lives in real time. And Chinese leaders think it’s the key to China’s future modernization and transformation to a global superpower.

Let’s talk about what concerns there are.

One of the biggest concerns that Chinese political leaders have when it comes to AI is political and social instability. They don’t want AI-generated content to act the way that social media or the internet broadly could, which is to disseminate information, maybe deepfakes, maybe viral news that could threaten the [Chinese Communist Party’s] hold on power.

For AI-generated content, it was about making sure that the models didn’t say the wrong things about sensitive topics. A lot of this is about social and political risk rather than about existential technological risk per se. 

From pretty early on, China has been developing a model registration system where basically AI services have to be vetted in advance and go through a battery of tests where they’re asked sensitive political questions and they’re supposed to either not answer them or say the “right thing” before they are officially registered. And once they’re registered, then they can be released to the public. So it’s a pretty tight regulatory framework for controlling things like political content.

We know that there’s a high unemployment rate in China, especially among young people, which is also a great concern of the government. Are people worried that this bullish approach to AI could affect employment numbers?

Yes and no. On the one hand, China is actually facing this demographic problem with a shrinking labor force. Fewer people want to go into, say, factory jobs, but China still wants to make everything the world wants to buy. 

They see AI as a partial solution to that, a way to automate things like manufacturing. On the other hand, though, there are growing concerns in China about job loss. And you see also some recent court cases in China where they basically are telling employers, you cannot use AI as a reason to fire workers. 

But I think overall, what they’re trying to do is they’re trying to find this balance between the diffusion of the technology and trying to prevent any kind of large-scale protest movement that might arise from sudden mass unemployment of, say, factory workers or food delivery drivers.

And President Xi was of course here in Washington last week. From the limited photos we have of the state dinner or of President Trump and President Xi on the tarmac, everything looks so chummy, even though we are so clearly in a cold war with China over AI. The president’s always talking about how he wants to win. 

Was there tension in his visit to Washington last week? Did you notice any?

The weird thing is that just between these two men, there seems to be this kind of romance where Trump really fawns over Xi Jinping. 

I think he has tremendous respect, even admiration, for Xi Jinping’s grip on power and control over the government apparatus in general. 

What is happening under the surface, though, is continuing tensions and a deep competitive structural relationship. That involves especially competition in AI. AI has emerged as a new fault line of competition between the US and China, one that could come to define the relationship going forward. 

I think that on the one hand, they’re talking about cooperation on things like AI safety and AI risk. On the other hand, there’s a whole bunch of issues where the US wants to lead. China wants to also get ahead. And when it comes to things like distillation or cyber capabilities, it’s as zero-sum as it’s ever been.

Who’s winning?

It depends on what you’re looking at and how you define winning. If you’re talking about the best models, for sure the US. And virtually everyone in China would agree, right? In fact, Anthropic’s Claude is very popular in China, even though technically Anthropic bans its use in China.

But aren’t their open-source models kind of popular here?

Yes, exactly. Chinese open-source models are getting more popular in the US, especially at a time when some of the capabilities are so incredible that most normal people don’t really need them, and they also don’t want to pay for the most advanced and most expensive models. Ultimately it’s this weird sort of cooperation and competition at the same time.

American models are popular in China. Chinese models are popular in America. We’re at each other’s throats, but quietly, politely.

Does the cold war make it all but certain that safety isn’t going to really be a chief priority for either country until something terrible happens?

The fierce level of competition and the very, very deep distrust between the two countries makes any kind of cooperation on AI safety very, very difficult. Even just talking to each other, having the US and China establish an official AI dialogue, which apparently they’ve agreed to, is seen as a huge first step, even though they’re just talking. 

There’s a long, long road to go before we can even think about something like an arms control treaty or a broader international agreement about AI and AI safety. But I think that’s what some people, especially in Silicon Valley, are trying to jump to.

If you haven’t heard of “live shopping,” you might soon. Livestream shopping is already a staple of e-commerce in China, where consumers log onto apps like Douyin and Taobao to buy products from a livestreamer — think QVC. Now, a new app is trying to bring it to the United States.

The Whatnot app features rolling live auctions that can last from three seconds to a minute. Users can bid on everything from trading cards to clothes to steaks. With about 550,000 hours of streaming per week, the site moves an extraordinary amount of product. 

Using Whatnot is fast-paced, and essentially frictionless. Users swipe at the bottom of their screens to place the highest bit for an object. If they win, the object is sent to their house, no confirmation screen needed. 

The system isn’t without its drawbacks. Hanna Krueger, a retail reporter for the Wall Street Journal, investigated how Whatnot has become an addiction for some users. She tells Today, Explained co-host Sean Rameswaram that she spoke to a user who spent over $1 million on the app — and lost everything in the process. 

Below is an excerpt of Krueger’s conversation on the podcast, edited for length and clarity. There’s much more in the full podcast, so listen to Today, Explained wherever you get your podcasts, including Apple Podcasts, Pandora, and Spotify.

Tell us about this app, Whatnot.

Whatnot is a live shopping app. Imagine the home shopping network had a baby with TikTok and eBay, and it was on your phone and it was running 24 hours a day. 

You open up the app [and] there’s countless numbers of categories. You can get coins. You can get cards. You can get horse tack, plants, goo, lobsters. I mean, the categories are expanding all of the time. And there are sellers live-auctioning off these goods in real time. 

There’s a countdown clock, and it’s counting down and the auctioneer or the seller is hyping up the chat, as they call it. And that chat is commenting like, “Wow, look at this. What a deal.” In the meantime, there’s some music. Sometimes it’s nice mellow music, other times it’s metal or EDM, depending on the personality of the seller. 

The end result is that the last person to swipe at the bottom of the screen, which is how you bid, gets the item. And once you have swiped and you’re the final person to do that, there’s no real confirmation screen. We’re just onto the next product and you only really find out what you got when it shows up at your door.

It started as a marketplace for Funko Pop figurines, which are these niche, squareheaded figurines in the collectible world, but it quickly expanded. I would say the biggest categories are sports cards and trading cards like Pokémon. And then the other one is women’s fashion and beauty products. 

I spoke to multiple people who found themselves just spiraling into holes of addiction and what they liken to gambling because of that “break format.” 

Tell us about the break format.

Break format typically applies to trading cards. That could be sports cards, Pokémon. It can also happen with rare coins, but the idea is that there is a sealed pack of cards and that sealed pack is pretty expensive to buy outright — say it’s $10,000. Before the show starts, they’ll say, “Okay guys, there are 10 slots that apply to this pack. So I need 10 people to buy in at a price of $1,000.” This is a steal, right? You’re getting a piece of the pie of a $10,000 pie for only $1,000.

The reason that they’re willing to do that is that within that pack, there might be cards that are worth $2,000, $3,000, $10,000. So you recoup quite quickly if you only paid a thousand for that. The other side, the flip side that often does happen is that the cards are worth nothing close to what you bought in for.

People will get a card that’s worth $50, $100, and at the end of it, all you have to show is these small-value cards. And those who have described kind of a cycle of problematic behavior say that when you lose, when you don’t make up the value that you paid, you just buy right into the next one until you get out of the hole. 

It’s this feeling of the high, the thrill, the adrenaline rush, the “Oh my God, look what I just did” when you get a card that’s way over what you thought. And there’s the despondency, this despair, the shame, and that feeling of, “Okay, I need to make my money back on that flip side.”

This is where this app really begins to look like gambling, to look like addiction, to look like a drug. As does the story of Sean Harding, with which you open your piece for the journal. Tell us about Sean.

Sean Harding is a 45-year-old finance manager from Denver. Sean found himself on this app and he wasn’t spending too much in the very beginning, maybe $100 a day. And then within 40 days he was spending over $1,000, $5,000 a day. Just swiping whenever he could get a free moment — when he was doing laundry, sitting on the toilet. 

It was only when his wife confronted him about this odd behavior, about some missing pages on his credit report, that he really looked at what he had spent and that number was crazy. It was $1.3 million in four months. He had borrowed money from a friend, he had taken out personal loans. And finally, when all that was depleted, he had used his employer’s credit card.

You spoke to the founder of Whatnot. What does he say when you tell him a story like Sean’s?

[Whatnot CEO and cofounder Grant] LaFontaine said that addiction does not register as a problem to them in their internal data, which they look at often. When we presented a story like Sean’s, they repeated that addiction does not register as a problem. When you breach the topic of gambling or people accusing this app of having gambling features, that’s where Whatnot definitely buttons up. They’re under pretty intense scrutiny surrounding that allegation.

They categorially reject that and they say they can’t comment on pending litigation. But they do say that they created an app that had the best user experience possible and that they’re new and they’re learning and they’re scaling and they’re assessing the marketplace in real time.

Are they implementing any guardrails to maybe prevent someone like Sean from ruining his life and his family’s life?

While they say that addiction doesn’t register as a significant problem, they also have introduced optional spending limits and watch time caps. They said that’s not because of addiction concerns, but because some users said that they spent more than intended. That was the distinction that they made. 

They said there’s other safeguards coming, such as “velocity of spend” alerts for sudden spending spikes. Some have PIN locking for categories, so you have to type in a PIN to go to a certain category. Some of these are self-policed. A watch time cap, you could presumably change that yourself, even though you’re the one that implemented it. 

They’re also saying that they’re going to develop some type of birthday reconfirmation option so they could catch underage users, which is coincidentally how I came to know about Whatnot. One of my colleagues’ daughters had spent over $5,000 in one night on Pokemon cards. And he only realized that when he looked at his credit card and was like, “What is this?” And he went to sell those cards at a card shop after the fact to try to recoup, and he could only get back like $300.

Is this just a Whatnot thing, or are there other ways kids and adults can be gambling their lives away on these live auction-type of platforms?

Live shopping has been one of the most predominant ways that Chinese shoppers shop for a long time, and the wave has crested over there a long time before it has here. It really started to take off and gain traction in America in the kind of post-pandemic world.

While Whatnot is definitely leading the charge, TikTok has their own thing called TikTok Shop that has a similar kind of setup. eBay is trying out stuff like this. And then for the card-collecting hobby specifically, Fanatics Live is a major source to get your cards and experience breaks and all that.

Do all of these different platforms, be it on TikTok or eBay, have the sort of gamified countdown clock to make it more intense, more exciting, and perhaps more addictive?

Whatnot’s found a way to create a really engaging and easy-to-interact-with atmosphere. They’ve also successfully merged the parasocial media aspect. There are relationships between the seller and the buyer, so if you keep going back to the same seller, they’ll remember your name, the people in the chat will remember. There are some folks who were like, ‘I found friends in here.’ 

Sean said that the night that he used his company credit card, he was reached out to by a seller and they were like, “Hey man, you got to join this. This break’s awesome. I have a great slot for you. I’ll give it to you at a discount.” And he’s like, “I don’t have any money.” And that’s when he realized that he had a little money. It just happened to belong to his employer.

That kind of shows that there’s this social atmosphere. Whatnot is definitely leading the charge, but other companies are seeing how successful this is, and I think it’s only time before they kind of merge all these aspects together in an attempt to take their market share from them.

Recently, while taking the metro to work, I encountered something that shocked me. The train was pulling up, just as the man in front of me tapped to pay to go through the gate, and a woman pushed past me to cut me in line. I still made the train (after I briefly considered acting in a way that would have made us both late for work), but as I passed each stop I got angrier and angrier. Why did she think her commute was more important than someone else’s? We learn to wait our turn when we’re children — why was that lesson lost all these years later? 

I’m not the only one irritated by this kind of thing. Almost half of Americans say that they’ve noticed people are ruder in the wake of the Covid pandemic. Whether it’s poor behavior at the airport or blasting music from your phone with no headphones in public, it’s like people don’t know how to act anymore. 

Kirsty Sedgman is a professor at the University of Bristol and the author of the book On Being Unreasonable: Breaking the Rules and Making Things Better. She argues that while people might not mind them as much anymore, manners are important. 

“They encourage us to think about others and to behave in ways that are not antisocial and potentially even dangerous,” she told Vox. “How can we have these conversations so that we figure out what we all want social life and public space to be like in a way that benefits everybody and doesn’t accidentally exclude [anyone]?

While some etiquette rules are pretty self explanatory (let’s not cut in line!), others are more difficult to navigate. “I always try to make a point and I’m not perfect, believe me,” Sedgman said. “I’m not saying that I know how to behave in every single circumstance and I’m always right. I’m no Emily Post.”

Post was the socialite who literally wrote the book on the subject: Etiquette in Society, in Business, in Politics and at Home was first published in the 1920s. More than a century later, her family is still in the business of teaching us how to mind our manners.

Lizzie Post is Emily Post’s great-great granddaughter and co-president of the Emily Post Institute. That’s a title and pedigree that could come with a lot of pressure. But Post says it’s all about your audience. “A lot of times, as long as I don’t get the sense that they would prefer a very formalized version of me, I try to reassure people: laugh, joke, swear, it’s okay,” Post said.

Audience isn’t the only thing that’s changed — the times have too. How has etiquette shifted over the years? And how do we navigate newer things — like our phones? We find out that and more on this week’s episode of Explain It to Me, Vox’s weekly call-in podcast.

Below is an excerpt of my conversation with Post, edited for length and clarity. You can listen to the full episode on Apple Podcasts, Spotify, or wherever you get your podcasts. If you’d like to submit a question, send an email to askvox@vox.com or call 1-800-618-8545.

For a long time etiquette has just been something that’s agreed upon. You have to have certain manners to socialize in certain circles and within certain classes. But I think that’s changed. Why do you think that’s happened?

My great-great-grandmother wrote from 1922 to 1960. While she talked about the standards of the day, she also allowed those standards to change. Think about 1922 to 1960 — there were huge swings in how people behaved. Chaperones were gone from the book by her second edition. Life was changing and evolving and she changed and evolved with it. 

Etiquette is a French word — it sounds fancy. I think that it can very easily have these connotations of being something that is about a secret society, or it’s only about formality, or it’s only for people who are very wealthy. It can also start to feel a lot like judgment. 

One of the reasons etiquette can feel so out of sorts, particularly right now, is because times are changing and they’re just changing really quickly. 

I think phone etiquette is really, really emblematic of that. One thing that is so common is people playing music or having their phone on speaker, no headphones. Do we just not know how to act when it comes to our phones? 

There are definitely rules of courtesy and blaring anything out of your phone, whether it’s a phone call, it’s a FaceTime call, it’s your TikTok feed, or that movie that you’re watching, is all inappropriate when other people are a captive audience and haven’t asked to be. 

It’s this mindful moment where people forget where they are. They’re so used to being in their own world with their phone six inches from their face that they forget that anyone can even hear what it is that they’re doing on their phone at that particular moment. 

It’s really hard to correct this behavior because generally, we don’t correct other people’s behaviors.

I admit, especially in public spaces, I am Team Mind My Business. But at what point do you step in?

It’s a really tough judgment call, and it’s going to be different for every situation. 

I think that when I am in a space that I’m only going to occupy for a short time, I am telling myself, “Mind my own business, don’t worry about it.” When it’s that train ride that we’re starting, or that four-hour flight that you’re on, I think we start to have to say something and to ask, “Hey, could you put some headphones on while you’re enjoying that or watching that or on your call?” 

I say, “Could you,” because they might not have headphones with them, but even if they don’t, it’s a reminder. It’s like that check of awareness that, “Oh, this thing I’m doing is impacting people around me.”

Here’s the rub: You can’t control other people’s behavior. You cannot control if this person is going to get defensive. If you’re going to ask the question, be prepared for rudeness in return. And I hate saying that, but it is the reality. We can’t control other people’s behavior.

So there’s clearly a wrong way to use speakerphone and FaceTime. What’s the right way to use them? 

We do a lot of talking on our speakerphones, even in our own homes. You might be moving from one room to another, passing through the living room where people are watching TV or you’re in the kitchen and all of a sudden a roommate has come into the room while you’re on a call that’s on speakerphone. You really want to let the person on the other end of the phone know that there’s other people present now. 

My cousin is so good about this. He always tells his daughter when she gets into the car, “Hey, Aunt Lizzie is on the phone. Say hi to her.” Then I know to watch my language. She knows to say hi to her auntie. And that’s just so much nicer than all of a sudden having a voice pop up and realizing, “Oh my gosh, I was just telling a really personal story and I didn’t realize there was a bigger audience for it.” 

What about having your phone on the table you’re at dinner? People feel very passionately about that one.

I am so bad about this one. I often have mine face down on the table. The funny thing is, I’m not looking for someone to call with an interruption, but it’s not good etiquette. 

It is really best to have that phone put away, to make sure that it’s on silent and it’s not going to interrupt your time together. To have it there, especially face up where any message that comes in is now going to catch the attention of everyone at the table, that’s an unnecessary distraction. 

Is there an obligation when you need to text someone back? Does it have to be within a certain time? You go about your day and the next thing you know, you look at your phone and you have 80 unread texts and it’s like, “Oh no, what do I do? My friends will all be mad at me.”

I wish there was, but it depends on the text itself. 

The ones I don’t think you should wait on is when friends or potential romantic partners offer a time to get together. Don’t let that sit. If there’s a reason why you’re letting it sit, talk to someone about that. Say, “Oh, I’ve got to check with my partner about that. I know we had talked about doing something else Saturday at that time.” 

Give them some kind of an understanding, because it’s a really beautiful thing to be invited to go spend time with someone. I think we want to treasure that a little more than we do in today’s age.

William Lawrence is a Democrat running to represent Michigan’s Lansing-area Seventh Congressional District with a campaign that largely focuses on data centers and affordability. It’s not an unusual story in 2026 — but not long ago, Lawrence was cutting his teeth in progressive politics as a co-founder of the Sunrise Movement, primarily focused on organizing around the climate crisis.

His arc, from progressive activist to politician, is emblematic of a bigger trend, as a wave of millennial progressives in midterm races across the country try to lean into issues that have broader, more immediate appeal with voters.

But that arc also raises big questions about the left’s changing priorities. In a recent interview for Vox’s podcast and video series America, Actually, host Astead Herndon spoke with Lawrence about his past activism and how he has pivoted. “It can sometimes feel like…folks hop from issue to issue, whether it’s climate, even anti-AI right now, without those things being solved,” Herndon told Lawrence. 

According to Lawrence, “that’s the reality of the time. It seems like everything is unraveling, and it is because the whole social, economic, political compact is being renegotiated, not just in this country, but on a global stage.” Still, Lawrence said, that earlier organizing shapes how he sees different challenges today.

The two also discussed whether opposition to data centers and AI has wide, regular-voter appeal in places like Michigan and how Lawrence went from hopeless and disengaged to regaining a sense of agency through organizing.

Below is an excerpt of the conversation, edited for length and clarity. There’s much more in the full show, so listen to America, Actually wherever you get your podcasts or watch it on Vox’s YouTube channel.

You started in Sunrise and were at the forefront of a young progressive movement that was aligned at that time — in 2018, 2019, and I think still — with progressive members of Congress like Rashida Tlaib or Alexandria Ocasio-Cortez.

I remember reporting about Sunrise at the time. The goal was to elect a Bernie Sanders, but the other goal was to have the party largely embrace things like a Green New Deal. How would you grade the efforts of the climate movement of that time, looking back?

I think we did some things very well.

We put climate jobs and justice at the top of the agenda for the incoming Biden administration. I think the Bernie-Biden task force is an often overlooked but very important chapter of that whole story, where Bernie and Biden both understood they had to bring their people to the table.

And that’s how Build Back Better got negotiated. We fought hard for Build Back Better, and ultimately came up two votes shy in the United States Senate.

Do you see that as a success or a failure of the climate movement?

I think Build Back Better was a success in coming to the table and negotiating, from the Green New Deal to Build Back Better, and then what we ended up with was the Inflation Reduction Act after it passed the final filter, which was Sen. Joe Manchin and Sen. Kyrsten Sinema. At the time, now, how I felt about it was, this is historic. This is the largest investment in green jobs and infrastructure in United States history. I know how hard it was to get here. I’m tremendously proud of the work that I did and so many others did.

Just in my district alone, we’ve delivered $26 billion in investment for manufacturing and sustainability because of the Inflation Reduction Act. But at the time, we knew that this is maybe 5 percent or 10 percent of what we need in scope. And now we’re experiencing the consequences of that.

Obviously, we still have a very volatile economy. We still lack energy security because we now have closed the Strait of Hormuz. We’re still too dependent on a global oil market, and we could use many more investments in solar and other kinds of renewable energy than we have now.

And ultimately, it wasn’t enough for the Biden-Harris administration to be able to win reelection, because people didn’t feel it enough in their pocketbooks.

Why do you think Democrats are talking less about climate?

I think it’s the intensity of the crises that we face. It feels like we’re falling through midair, after the end of all of the constitutional protections, the rule of law, and the economic security that people had previously enjoyed. 

There are a lot of very acute and urgent crises, and I think that that has bumped climate, which is a super massive but also somewhat slow-moving crisis, a little bit lower on the priority list just in terms of what comes up for people on the Hill.

You went back to Michigan to focus on housing and advocacy. Did that make you think a little differently about the national work you were doing?

I think it’s fair to say that when I was younger, I got very focused on this climate issue, and I was very focused on it because it is this existential crisis we all face.

But it was after the pandemic that I wanted to bring my work closer to home. I wanted to do the work of putting down roots in the place that had raised me and make my life there. And in seeking to organize there, it was clear that we needed an in to organize that was more immediate.

“I believe in the agency of regular people taking charge of our own lives and working together to build a better future.”

More immediate than the Green New Deal.

Than the Green New Deal or than the looming climate crisis. People know that that’s an issue, but they’re like, “I’ve gotta pay rent at the end of the month.” And as an organizer, I believe that you have to listen to people, you have to hear what’s going on for them, and then work on issues that have the potential to bring people together.

Housing is one of those issues. Whether you’re in the city of Lansing, whether you’re in small towns, urban, rural, suburban, young or old, everybody can agree that the rent is too damn high and it’s too hard to buy a home anymore.

Were you surprised Trump won the state in 2024? 

No.

Why not? 

Because I saw it all unfold. We hoped to deliver more through the Biden-Harris administration than we ultimately did through Build Back Better. We didn’t deliver as much to relieve the economic anxiety that people were feeling. And then I do believe that the war and the genocide in Gaza and our blank check to allow that to happen with our money and our tax dollars destroyed the Democratic coalition that had been assembled from 2017 through 2020 for the purpose of beating Trump.

I just saw that unfolding over the course of the whole year, just like a slow-motion train wreck.

I want to ask locally, what do you think is most important in this race: housing, data centers, or pushing back against Donald Trump?

If I had to choose, I’d say data centers because that’s where people in the district are finding some common ground across the spectrum — in fighting back against the concentrated power in Silicon Valley, which has been fully enabled by [incumbent Rep.] Tom Barrett and Donald Trump.

So you’re saying that’s the issue that allows you to create cross-ideological coalitions?

Yeah.

Our last question is, just thinking through the scope of your career and the kind of progressive work: It can sometimes feel like folks hop from issue to issue, whether it’s climate, even anti-AI right now, without those things being solved, right?

Like, we still have a climate crisis. The rent is still too high. All of these things are still happening. How am I to trust that AI is the crisis of our time when I’ve heard so many crises of our time?

Some have called it a polycrisis — overlapping crises.

That’s the reality of the time. It seems like everything is unraveling, and it is because the whole social, economic, political compact is being renegotiated, not just in this country, but on a global stage. I don’t think it serves us to lie about that. I mean, here’s the reality, folks.

Everything is up in the air. Does it feel like you’re falling through midair? It’s because we are. All the old protections, all the old agreements have been chopped up. So we have got to come together. This is what I believe in more than anything, if there is a through line…I believe in the agency of regular people taking charge of our own lives and working together to build a better future. 

That’s what I learned as an organizer, and that’s what gave me hope actually, at a time when I was doomer-ing in my younger years. And I told you, I wanted to just drop out of society and all that. It was seeing that regular people have come together before, and they’ve won a better future for their kids and grandkids, that gave me a path to follow in.

It does feel like a powerless time, I have to tell you. And a lot of this AI revolution, to your point about control or about agency. We’ve talked to so many people who just think that these things are happening to them, rather than feeling that they are participating in the changes afoot.

But you’re telling a kinda hopeful story. 

I’m a hopeful guy. I was angry too, even when I got into this race. I was angry especially about the wars. But then in campaigning, I’ve met so many people across the district, old friends, new friends, and they give me a lot of hope because you just see the way that people are already performing miracles on a daily basis.

Maddie Kowalski is a student at the University of Florida. Last November, someone found nonconsensual intimate images and videos of her, stitched them together, and uploaded them to X, where they were viewed millions of times.

“Can you imagine?” asked Ej Dickson, a senior culture writer at Wired who reported on Kowalski’s story earlier this month. “You’re 19 years old and a video that you don’t even remember shooting, let alone consenting to, goes viral nationally along with your name and your sorority, and your parents, your grandparents, everybody in your life sees it. It’s a humiliation and a privacy violation that is kind of incomprehensible.”

The video was followed by a campaign of intimidation and harassment. Kowalski and her parents received threatening phone calls and texts. People on the internet posted old photos of her, her father’s LinkedIn, and even his golf handicap. Kowalski sought help from the police, from the university, and from X. But the video was being shared faster than the posts could be removed, and her name became a trending topic on X.

What made the video go viral, Dickson explains, is something called the “burnerverse”: a loosely connected network of anonymous “burner” accounts on X. The community mainly exists to discuss college sports, but sometimes it disseminates videos and rumors aimed at shaming young women and ruining their lives. The burnerverse has harassed Kowalski and her family for nearly a year, sometimes encouraging her to kill herself.

Today, Explained co-host Noel King spoke with Dickson about the online network that targeted Kowalski, and why it’s been so difficult to do anything to stop it. 

Below is an excerpt of the conversation, edited for length and clarity. There’s much more in the full podcast, so listen to Today, Explained wherever you get podcasts, including Apple Podcasts, Pandora, and Spotify.

Your story describes how Maddie has, without even realizing it, entered into something called the burnerverse. This is something I think a lot of us, prior to your piece, had not heard of. What is the burnerverse?

I hadn’t heard of it either. The burnerverse is an online community of anonymous accounts, and ostensibly these are college sports fan accounts. They talk about the coaches, the players, new recruits, they share memes after their team loses, but there’s a corner of the community that has emerged that is pretty violently racist and misogynistic and antisemitic. And one of the things that they’ve started doing in recent years is doxxing young college-age women like Maddie.

Why is it called the burnerverse?

Because it consists of burner accounts, anonymous accounts.

Okay. So the reason that whoever is spreading these videos of Maddie and others feels comfortable doing that is because they’re anonymous. Nobody knows who’s behind them?

Yes, exactly. And I also want to be clear: This is a small segment of the college sports burner community. 

There are a lot of these accounts that are pretty innocuous and would never dream of doing something like this to a young woman. I would say probably the vast majority of these accounts are not like that, but it is a sizable enough segment that they’re able to make these types of narratives go viral.

What do we know about who is part of the burnerverse? Who is likely to be a part of this?

That was one of the most shocking things that I learned from my reporting, actually. I had assumed that these were largely frat kids. I assumed that they were largely SEC frat kids because a lot of these young girls who have been doxed were students at SEC schools. But when I actually started looking into these accounts and who was behind them and who was sharing these images and making these jokes about Maddie, I learned it was way more wide-ranging than that. 

There are a lot of college students who are involved in the burnerverse, but it’s also men who are 10, 20, 30 years out of college. Sometimes they have wives. Sometimes they have children. Sometimes they have grandchildren. They’re from all over the country. It’s not limited to one race or economic demographic. It’s really very wide-ranging, which is kind of what makes it so terrifying.

And it is not just Maddie Kowalski. What did you learn about other young women who are targeted by this?

No, it’s not. So perhaps the most well-known case is this girl named Mary Kate Cornett, who was a student at Ole Miss in February 2025 when this salacious campus rumor about her sex life started going viral. And the rumor was false and it was clearly defamatory, but it went so viral that it kind of escaped containment from Ole Miss and went viral among SEC schools and then went viral nationally, to the point where her name was trending nationally on X. 

This very famous ESPN personality, Pat McAfee, talked about her without using her name, but talked about her and the details of this rumor on his podcast. And a lot of these athletes and college sports personalities tweeted about it as well. What started as an unverifiable and false rumor about a college girl’s sex life was just blown up and amplified to the degree that it was a national news story within a couple of hours. 

The speed with which this happens — [Cornett]she was telling me that basically a few hours after she learned that she was going viral, her parents’ house was swatted. The speed with which this community works toward making these women’s lives a living hell is really, really rapid.

Did you come away with an understanding of how an online community that is meant to be talking about college sports descends into this?

Part of it is just a consequence of how internet subcultures work. Even fairly innocuous, large internet subcultures and communities, there’s always going to be a small part of any online subculture that splinters off and forms its own community that is going to skew a little bit more extremist. 

That’s not surprising in itself, but I do think that there is something about the discourse surrounding college sports, particularly the media ecosystem that’s represented by Barstool, that does create a space for this type of rampant misogyny to foment. 

I don’t think it’s a coincidence that some of the accounts that I have seen posting really horrible defamatory content about Maddie are smaller Barstool accounts that are associated with individual colleges. I do think that has definitely played a role.

Why did Maddie become the target of these people in the burnerverse?

Because she’s young and she’s attractive. Basically, the criteria is these women are young, attractive, they’re usually white for some reason, and they are perceived to have departed from a sexual norm in some way. 

They’re either subject to some sort of rumor about their sex life or their sexual behavior or maybe a text message leaks about them or a photo. Then these guys take it and they blow it up and it goes viral nationally. 

They’re very purposeful about it too. I can’t tell you how many tweets I’ve seen that are just tweets of a girl’s name and they’re like, “Make her famous, make her go viral.”

In an ideal world, this kind of thing would be stopped before it even happened. So, knowing that there is not much that victims can do, what are the methods of preventing this?

That’s a very complicated question. I know that there are platforms developing various AI tools in order to prevent nonconsensual intimate images from spreading. But I imagine that would be pretty difficult to regulate, especially with the rise of deepfakes and with the rise of-AI manipulated nonconsensual images, so I don’t have a lot of faith in that. 

Honestly, I think that the biggest path toward prevention is — can I say this? Parents raising their sons to not be pieces of shit.

You can say that. You can absolutely say that.

I think that the biggest course of prevention is parents raising their sons to not be pieces of shit, honestly. To just set a clear line and say, “It’s not okay to view women as commodities. It’s not okay to talk about women like this. It’s not okay to view social media engagement as the ultimate goal without thinking about the lives that you’re impacting by spreading this content.” I think that parents need to have conversations with their kids about that. 

I think that there needs to be education within fraternities and sororities and universities because it’s not like what happened to Maddie Kowalski is an isolated incident. I was able to identify almost a dozen other young women who have been targeted by the burnerverse in this way. 

I think that there needs to be more discussion of this in general because it’s not just the burnerverse. It’s this larger ecosystem that is socially ordained at this point of men saying horrible things and doing horrible things to women with impunity. That’s the real problem.

Six months ago, first lady Melania Trump walked into a White House education summit next to a faceless robot powered by artificial intelligence, which she said could one day give children personalized instruction.

“Imagine a humanoid educator named ‘Plato,’” she said. “Access to the classical studies is now instantaneous…humanity’s entire corpus of information is available in the comfort of your home.” 

Weeks earlier, her guest at the State of the Union had been a 10-year-old from Alpha School, a fast-growing private school network where children learn reading and math from AI software for two hours a day, and spend the rest of their time on workshops and projects overseen by adults the school calls “guides” rather than teachers.

The Trump administration had chosen AI as the future of American schooling, and wanted everyone to know it. But they picked a bad year to ask Americans to trust the technology with their kids.

Earlier this month, top leaders at Anthropic, OpenAI, and Google DeepMind publicly urged for an AI development slowdown, warning that companies racing to build ever more powerful systems could lose control of them, threatening not only large swaths of infrastructure but even humanity itself. Trump responded by calling the panic a “hoax” and said the US would not “hinder or stifle” the industry, because “whoever wins AI wins.” This “let them cook” mantra has met nearly every AI fear this year, from job loss to data center construction.

Yet the public is not convinced, and parents perhaps least of all. Efforts to bring AI into classrooms have run into organized opposition, and much of it is coming from inside Trump’s own coalition, setting up a fight between the administration’s pro-tech allies and conservatives who think screens have already done enough damage to kids.

In the nation’s two largest school districts, New York City has recently pulled student-facing generative AI from classrooms through eighth grade and Los Angeles blocked it on all student devices. But it’s not just Democratic-run cities that are pushing back. Earlier this year Utah barred students from using generative AI on assignments without a teacher’s say-so, and required schools to notify parents when it’s used in class. Oklahoma followed suit with a law, passed without a single “no” vote, that requires districts to tell parents which AI tools they use and what data they collect, and lets families opt out. Florida went further last week, when its Board of Education adopted rules for parents to opt their children in before they use AI at school, guaranteeing non-AI alternatives, and banning “companion” chatbots outright. 

Even at the Heritage Foundation, a conservative think tank with close ties to Trump, leaders are warning about the risks of AI in the classroom. In a March report, Heritage officials cautioned that AI tools built with addictive design features can harm children’s attention, mental health, and learning, and said schools should set limits rather than assume the technology will regulate itself. In July, the Manhattan Institute, another conservative think tank, recommended New York State bar students from using AI until third grade and keep it out of tests at every grade.

All the while, the Trump administration continues to push in the opposite direction, casting the wider AI race as one the US cannot afford to lose to China. Last year, Trump signed an executive order creating a White House task force to bring AI into K-12 classrooms, train teachers to use it, and build an “AI-ready workforce” to keep America as the global leader.

Polling data released in August found voters across the political spectrum concerned about AI in classrooms, with 85 percent worried about students using it to finish assignments without learning anything. Republicans were, if anything, slightly more eager than Democrats for government guardrails, 80 percent to 76 percent. Put differently, Republican voters appear closer to the Heritage Foundation than to the White House, which points to a coming clash inside the GOP over parental rights, innovation, and whether schools that take public money should have to prove their tactics work.

Up to this point much of the AI backlash has focused on regulating the tech in public schools, while some of its most ambitious boosters have concentrated on building private schools and developing new tools. A state can cap screen time in its elementary classrooms and let another family down the road pay thousands of dollars for Alpha, and everyone can call it a win for parent choice.

That peace may start to fray though as more public money comes on the table. Texas recently expanded vouchers to Alpha network schools and the company has also sought to open publicly funded charter schools. Other states are likely to confront these issues as well, meaning the AI education fight on the right may soon be unavoidable — and would land before an electorate whose patience for the tech, in schools and increasingly everywhere else, has been running thin all year. 

The Trump administration’s AI bet

The AI-in-schools executive order came just three months into Trump’s second term, and called for AI education to begin as early as kindergarten. Within five months, Education Secretary Linda McMahon was touring Alpha School’s Austin campus, describing it as “the most exciting thing” she’d seen in education in a long time. 

Alpha is hardly the first education company to promise a shiny new school model, but it has quickly become an influential player in the growing national debate around AI in schools. Its story so far reflects many of the big questions around AI in the classroom, including how to determine if the tech is helping or hurting kids, who is profiting, and what role governments should play in facilitating its adoption.

Alpha had been a small operation for most of its existence. MacKenzie Price, a Stanford graduate, co-founded it in 2014 after her second-grade daughter complained that school was boring. The tech billionaire Joe Liemandt became Price’s partner and the school’s principal financial backer. As recently as 2024 it served about 300 students across three campuses. 

Growth exploded in summer 2025, when Alpha took over campuses from a shuttered Montessori chain and announced plans for 10 new schools. This August it announced plans for 27 more.

Behind the school’s growth is its claim that students enrolled can cover a full day’s worth of academics in just two hours with AI, learning twice as fast as they would at a traditional school. Alpha can “enable the increase in human intelligence,” Price declared on a panel in Washington, DC, earlier this month, because “our academics can be delivered so much more efficiently and effectively.” 

Alpha says results from MAP tests, a widely used measure of academic growth, show its students learning 2.6 times faster on average than similar students at other schools. But no outside researcher has audited or independently verified those results.

That may soon change. MIT’s Blueprint Labs, the group behind some of the most rigorous charter school research in the country, told Vox it is designing pilot studies to measure the effects of the school’s programs on achievement and engagement, using experimental designs. Alpha is paying for the research; the lab says its own researchers will set the questions and methods. Harvard’s Human Flourishing Program is separately planning to study students’ life skills and well-being via a longitudinal study during the next academic year, Christina Hinton, who is involved with the research, told me. For now, Alpha’s academic claims remain its own.

Meanwhile, scrutiny of Alpha has grown. A Wired investigation last year described the school’s surveillance system as capable of recording students’ screens, keystrokes, and mouse activity and potentially tracking their eye movements. A 404 Media report published this winter claimed Alpha’s own internal documents said its AI-generated lessons sometimes did “more harm than good.” Alpha has disputed both reports, saying they relied on cherry-picked examples and accounts from fired employees, and provided me with an open letter it says every current family at the Brownsville campus signed, stating that the school Wired described doesn’t resemble theirs.

But education officials are still wary. This month, ProPublica and the Texas Tribune reported that at least nine states, Texas among them, had turned down Alpha’s bids to open publicly funded charter schools, with officials repeatedly pointing out that the model had only been tested on affluent students. (Tuition varies based on location, ranging from $10,000 a year at its Brownsville campus to $75,000 at its most expensive California schools.) Where Alpha had worked with lower-income students, ProPublica and the Texas Tribune found, it either hadn’t released enough data to back its claims of student growth or state test scores fell well short of what its leaders had predicted. 

After the Texas State Board of Education rejected Alpha’s charter bid in the summer of 2025, Alpha didn’t reapply. While policy advocacy was “frustratingly slow,” Price wrote, Alpha’s goal was “for students everywhere at every income level to be able to use this model of learning.” They turned instead to Texas’s new voucher program, which lets families spend public money at private schools. Unlike a charter, a private school in the program doesn’t have to give state tests or show anyone how its students perform. The comptroller, which runs the Texas Education Freedom Accounts (TEFA), approved most of Alpha’s Texas schools, which, counting virtual programs and specialty academies, now number more than 30.

None of the concerns that Texas’s State Board of Education had raised about Alpha’s test scores, missing data, or AI applied. “That wasn’t relevant,” Travis Pillow, a spokesperson for the TEFA voucher program, told ProPublica. “Our role is to make sure that a school meets the requirements in the law.” 

The five most expensive Alpha schools in Texas opted to leave the voucher program earlier this month, but many Alpha network schools are still participating, including Texas Sports Academy, Montessorium, NextGen Academy, Nova Academy, and Waypoint Academy. TEFA confirmed to me that students are attending Alpha affiliates via vouchers this year. 

Anna Davlantes, a spokesperson for Alpha, told me that in lieu of vouchers for their most expensive campuses, her network is offering families financial aid. 

The conservative revolt

Despite Alpha’s enthusiastic embrace in the White House, the introduction of AI in schools is increasingly challenged by forces on the right, from grassroots MAGA parents to social conservatives and technocrats.

The new AI rules in Florida, Oklahoma, and Utah, for example, grew out of a broader conservative push against screens in classrooms that predated the rise of chatbots and was turbocharged by parents’ frustrations with virtual learning during the pandemic.

Earlier this year Iowa capped elementary school screen time at an hour a day in a bill Gov. Kim Reynolds signed with Health and Human Services Secretary Robert F. Kennedy Jr. at her side. Tennessee passed a weakened version of a bill that would have banned devices in elementary schools outright, while similar bills in Missouri and Kansas died after school districts and teachers unions objected.

Uniting these efforts is a growing conviction that the last 15 years of education technology has failed most kids. The sponsor of Kansas’s bill used to sit on his state’s board of education and remembers thinking that ed tech held such promise. “What we found out is that not only is it not the education of the future, but it is destroying the education of the past,” he told Education Week. A growing number of lawmakers and parents have been citing The Digital Delusion, a book self-published in 2025 by neuroscientist Jared Cooney Horvath, that blames the proliferation of screens for declining student achievement. In January Sen. Ted Cruz (R-TX), who is otherwise one of the Senate’s leading opponents of AI regulation, invited Horvath to testify about the harms of AI on student learning and mental health.

The screen skeptics overlap with a more traditionalist wing of the GOP that objects to classroom technology on principle. Missouri’s original bill would have required cursive and kept 70 percent of student work on paper. The classical education movement, which holds up teachers as a moral example, has grown fast on the right. (Of the 895 identified classical schools nationwide, roughly one-third opened or converted to the model between 2020 and 2024, according to a Heritage Foundation tracker.)

The strand of conservatism that cuts closest to Alpha School comes from a third group, the school choice advocates. On paper, Alpha sounds like what the school reformers have wanted for years: It can offer an alternative to underperforming local public schools, its AI programs can theoretically compensate for inconsistent teacher quality, it provides personalized instruction, and it checks mastery on every lesson, so families can easily know whether their child is learning. Those are problems the education reform movement was launched to solve, and Alpha claims to fix them all.

The reformers, though, want proof. Mike Petrilli, who runs the conservative Thomas B. Fordham Institute think tank and has spent decades advocating for charter schools, wrote recently on his Substack that parents who want to pay for Alpha out-of-pocket should be able to, but “that doesn’t mean the rest of us shouldn’t be skeptical, or that we shouldn’t worry about fraud, or that we should allow public funding to flow to these school.”

This split was on display in DC this month at the education conference where Price appeared. Molly Hart, who runs Utah’s public schools and calls herself a school choice supporter, argued that once a school takes taxpayer money, “the game changes,” and parents can’t make a real choice without data about how it performs. Erika Donalds, who chairs the education center at the America First Policy Institute and is one of the Trump administration’s closest allies on schools, said parents, not the government, should decide which results matter. Donalds compared the choice to picking a restaurant on Yelp. 

The evidence has started to catch up with parental concern. In a randomized trial involving nearly 1,000 students at a Turkish high school, those given access to a GPT-based math tutor that readily supplied answers scored 17 percent lower on a subsequent exam taken without it. A version designed to withhold answers eliminated that penalty but produced no statistically significant learning gain. A newer working paper tracking nearly 27,000 Chinese students found that adopting AI raised homework scores by 18 percent but lowered monthly exam scores by 20 percent within six months.

Alpha School is working to add more detail to that pile, and other studies will surely follow. It’s at least possible that some aspects of AI in education prove more promising with further study. But public skepticism will remain a major barrier, either way. On an issue where parties are still sorting themselves out, the White House and its allies are swimming against a growing anti-tech tide that extends far beyond just AI or even schools. That’s unlikely to change before 2028, and whoever wants the Republican nomination will have to say which parents they’re listening to.

When the AI apocalypse becomes regular dinner-table conversation, you know we’ve reached the freakout stage. Rogue AI agents are hacking into competitors. Former Big Tech employees are posting Skynet-style warnings on social media. CEOs are crying for help in ways that have made this once-wonky tech issue a frontline political fight. 

I still think many of the concerns outlined by the AI leaders themselves are massively self-serving — especially when their companies are soon to go public. But the growing list of AI scandals is real. And it tells us that the status quo for regulating this powerful new technology is not working.

The question now on everyone’s mind: Is any kind of global brake possible? What form of AI safety regime is practical, over the short term, when there is no trust between major governments? And what could possibly work when companies are loath to give up their secret sauce and are bent on domination in the global AI race?

Two major conversations are now playing out, in real time. On Wednesday, President Donald Trump welcomed Chinese President Xi Jinping to Washington for a three-day summit — part of which will be dedicated to AI risks. The same day, with the United Nations meeting for its annual General Assembly, heads of the world’s leading AI companies urged the UN to police the emerging technology, or risk potentially wiping out humanity. “We could lose control of the future to AI,” OpenAI’s boss, Sam Altman, bluntly told the UN’s Security Council.

Confronted with this collective freakout, it’s time to take a long, deep breath. 

In fact, AI safety has been on a lot of policymakers’ minds for years now, gaining momentum after ChatGPT was introduced in 2022. Governments have held hearings, convened expert groups, and drafted their own outlines of how to keep it under control. These weren’t just idle listening exercises. They aren’t all well known to the wider public, but they yielded real plans.

Barring any breakthroughs this week, we already have a quasi-planetary shield against the potentially runaway technology. That’s the good news.

The bad news is that it’s not really up and running yet. It’s also not clear whether it will actually work. So what is it — and how can we fix it so we have a global system that responds in time to deal with the incredibly fast-moving threats from AI?

What the global AI patchwork looks like now

For the last four years, the most tech-savvy nations — and would-be AI leaders — have been running serious conversations about this exact AI safety threat. Some are in Congress and the White House; some at the UN, at the G7, and in other capitals.

I’ve been covering this closely as a global technology journalist, from my current perch at a think tank. Here’s what the landscape looks like:

• There are national AI safety and security institutes, government-funded bodies whose job it is to kick the tires of AI companies’ latest products before they are let loose into the wild. 
• There are also voluntary commitments by companies, many of which have made their own pledges or developed joint standards, to protect elections from AI threats, stop the spread of AI-fueled deepfake imagery, and joint government-corporate efforts to bake safety into how the technology develops.
• At the international level, there’s a G7-led reporting mechanism — embraced by the most important Western tech nations that allows AI giants to share how they are building their latest models, as well as create standards for how to reduce catastrophic risks.
• An international scientific report provides a yearly update on risks posed by the most advanced AI systems, based on existing research, to help governments plan for the worst. 

What’s missing from all that?  What we currently lack — and what is needed between now and the end of 2026 — is a way to turn this cottage industry of AI safety mechanisms into a functioning, but crude, first-responder system when things suddenly go wrong. 

The last four years have laid out a pathway, and some of the necessary systems even exist. But there’s no way to respond globally, and in real time, when an AI crisis hits — especially if such a possibly doomsday event cuts across countries already skeptical of each other.

This week’s US-China summit may be a step in the right direction. Under proposals outlined by American officials, Washington and Beijing could set up a hotline between US Treasury Secretary Scott Bessent and Chinese Vice Premier He Lifeng in case of an AI incident affected each country’s national security. 

It’s still unclear if Trump’s meeting with Xi will lead to such progress. Chinese officials also have balked at Washington’s pleas for AI rules because, so far, Congress has failed to act, and China already has some of the world’s most stringent AI oversight.

For AI to be “safe,” this conversation will need to go beyond this week’s US-China summit. Relying just on Washington and Beijing — arguably the most important AI powers — would not solve the underlying problems, nor would it make other countries feel more comfortable.

What we need next, so it really saves us

I’ve been talking to AI safety experts and government officials, and it’s clear the missing piece is a way to activate this whole system in a crisis. Countries don’t all have to have the same AI safety policies, and they never will. But as with nuclear weapons — a similarly high-threat technology that the world found a way to contain — safety requires a rough global agreement on how to respond quickly when something goes terribly wrong.

What’s needed right now is a 90-day, opt-in rapid response mechanism that joins existing pieces of the AI safety puzzle together. I’ve pieced together some ideas about how it should work, and who needs to sign on. Granted, none of what I outline below is sufficient. But, together, they are more plausible than trying to negotiate a comprehensive global regime — let alone arrange another AI summit — while Washington, Beijing, and other national capitals disagree over what “AI safety” actually means.

All of these options are based on existing mechanisms, are derived from efforts that have worked in other policy areas, and provide a band-aid to the AI safety dilemma until a more durable solution can be negotiated.

• First, developers, AI safety institutes, and regulators, from across different countries, should agree that when an incident occurs, it triggers a specific Chernobyl-style protocol. That would mean submitting a confidential report within a 72-hour period to a national designated responder and a small technical secretariat. It can build on the OECD’s AI Incidents Monitor, lead to a technical, multi-stakeholder confidential investigation into what went wrong, and, subsequently, the publication of an anonymized lessons-learned note.
• Second, governments and developers can agree to publish common declarations related to safeguards, residual risks, escalation thresholds, and independent auditing before the release or upgrading of a next-generation model. It would use the G7 Hiroshima AI Process reporting framework as its base, and turn the current hodgepodge of corporate safety declarations and voluntary standards into a common minimum disclosure obligation. View it as similar to the collective bank stress tests after the 2008 global financial crisis. It can allow outsiders to truly compare models’ safety protocols without putting someone in charge of determining which company is doing it best.
• Third, create a Cold War-style US-China AI safety hotline — but see it as an initial step that can later be opened to other countries. There are good reasons, in the long term, that Washington and Beijing should not be allowed to rule artificial intelligence between them, and the rest of the world should have a real seat at the table. But for now those are the two “great powers” in the tech conversation, and we can set that philosophical argument aside to create a standing, technically-informed direct line of communication for acute AI-risk incidents. Its remit is inherently narrow: prevent dangerous misunderstandings linked to serious model incidents, AI-enabled cyberattacks and/or alleged breaches of agreed safety commitments.

There are obvious limits to what I describe above. For one, it’s an inherently Western-centric view that primarily discounts global majority countries. It also places too much sway on existing institutions like the Organization for Economic Cooperation and Development, as well as on the US-China relationship. Other countries’ officials will legitimately balk at all three options, and rightly so.

But this is not about creating a vague, unenforceable UN-led mandate for AI safety. Nor is it about corralling the geopolitical cats to hammer out a global AI treaty. The options — a collective safety protocol and incident reporting protocol; common pre-release standards; and a US-China AI safety hotline — are inherently short-term. They are also based on existing efforts and those that have worked successfully for other policy areas.

There will be time to quibble about the future of AI safety. But now is not that time. The latest AI models are moving faster than many had expected, tech bosses are worried their creations are already out of control, and the window for action may be smaller than we all think.  

What is required are practical steps to assuage people’s growing concerns amid heightened geopolitical tension, a lack of trust between governments and companies, and a need not to let the perfect get in the way of the good.

The AI freakout of the past few weeks has centered on nothing less than the risk of human extinction — on the possibility that artificial intelligence could wipe us all out. But the reality is, AI programs are already having a direct and sinister effect on people’s physical health: UnitedHealth Group, the largest health insurer in the US, has been accused of using AI programs to deny payments to people, despite their doctors saying the care was necessary, and is facing a class-action lawsuit. 

Humana and Cigna are facing their own litigation over similar allegations. Meanwhile, Medicare has launched its own AI tool that will help the government determine whether to approve prior authorizations, and doctors are sending the federal government reports of patients at their offices who are in tears because they can’t get the medicine to manage their chronic pain.   

The claims in the legal complaints are harrowing: Families are desperate for an elderly loved one who recently experienced a serious fall to be allowed to stay in a nursing home. Doctors are flummoxed when a patient who may need months of rehab is cut off by an algorithm after a couple of weeks. 

This is AI’s threat to people’s health right now. 

While the current conversation may be fixated on something more Hollywood, private insurers and the Trump administration are already deploying AI programs to vet people’s health insurance claims — and the evidence is growing that these programs are not always doing a good job and are instead putting people’s health at risk. Until these lawsuits are settled or policymakers can figure out a way to regulate how insurers use this new technology, it will be up to each of us to make sure that an AI program isn’t being used to cheat us out of insurance benefits that we’re entitled to.

“The type of care that clinicians deliver is very sensitive to the patient’s unique circumstances. That may not really show up as a number. There’s not a numeric value that tells you the completeness and connection of a patient’s social network,” said Mika Hamer, a health services researcher at the University of Maryland. “There really are a lot of factors that really can’t be quantified and so aren’t really suitable to go into these models. If we start to rely exclusively on the models, I think we run the risk of losing that human touch.”

Patients say private health insurers are using AI to deny claims

Insurers and patients fighting over medical claims is nothing new: Policyholders want the most coverage they can get and payers have a business incentive to pay as little as they can. Health insurers have also used algorithms for decades: Plug in somebody’s age, their weight, whether or not they smoke, and one of the old-fashioned algorithms could spit out a rough estimate of how much medical care they would need before they died. Simple — and comparatively transparent. 

When you submitted a claim or sought prior authorization, there was supposed to be a human doctor who could review all of the relevant information on behalf of the insurer and make the final decision, in theory with the patient’s best interests in mind. It didn’t always work that way in practice, of course, but that was the idea.

What’s different right now, according to Daniel Schwarcz, a law professor at the University of Minnesota who has looked at health insurers’ AI use, is that these newer programs can make very specific predictions about someone’s future health based on the information that’s fed into them, and those predictions are being used in the process of approving or denying medical claims. And exactly what went into making those predictions — and whether it is the AI or the human who ultimately holds the final authority — is often opaque from the outside.

“The reason we want human review is we want human substantive judgment. We want AI to be supporting that,” Schwarcz said. “But the difficulty is distinguishing between AI supporting meaningful human processing and AI basically replacing the human with the human just passively defaulting to the AI judgment. It’s just incredibly hard to distinguish between those two in practice, particularly in rules or regulations.”

And those questions are at the heart of the lawsuits brought against three major insurers in the past few years about their use of AI in processing claims.

In May 2022, Gene Lokken, then 91 years old, fell in his home. He was taken to the nearby hospital in Tomahawk, Wisconsin, and treated for a broken leg and a broken ankle. Less than a week later, he was placed in the health system’s hospice care because, according to the lawsuit filed by his family, he was “very weak, not communicative, and in constant pain.”

But a month later, after receiving skilled nursing care, Lokken started to improve. By late June, his doctor had removed his splint and placed his leg in a removable boot. In July, he started physical therapy to get up and walk again. And for a while, his Medicare Advantage plan sold by UnitedHealth covered all of the services he needed.

Until July 20. Around that time, according to the lawsuit, UnitedHealth denied any more services at the nursing facility where Lokken had been for the past two months. It insisted that he could be safely discharged and more time at the nursing care facility “was not medically necessary.”

The insurer’s decision left Lokken and his physician “dumbfounded.” The physical therapist notes from the time said Lokken was still suffering paralysis and weakness after only a couple weeks of rehab. His family appealed, but UnitedHealth continued to deny the claims, asserting that because Lokken could feed himself and did not require help to clean himself, he did not need nursing care and could go home. 

From July 2022 to July 2023, Lokken remained at the nursing home, but his family had to pay between $12,000 and $14,000 monthly to keep him there. And on July 17, 2023, Lokken died — having never left the facility.

The lawsuit alleges that the decision to stop covering Lokken’s nursing-home stay was largely made by an AI algorithm, called nH Predict, which UnitedHealth follows so unquestioningly that employees were allegedly at risk of being disciplined or fired if their claims decisions went against the AI’s projections. The complaint alleges that the AI model would typically start denying claims after two weeks of nursing home care — and the attorneys have more examples than Lokken.

Glennette Kell required emergency surgery after a fall in late August 2023 and was referred to a nursing home rehab center. By mid-September, UnitedHealth was denying her claims for nursing care, despite the fact that, according to the lawsuit, she still could not walk. Kell had to pay $10,000 of her own money to cover the rest of her bills. Darlene Buckner suffered a heart attack in October 2023 and underwent triple-bypass surgery. She was readmitted to the hospital in late November because she was experiencing complications, but less than two weeks later, UnitedHealth allegedly denied any further treatment at the hospital because it was deemed medically unnecessary. At the time, according to the class-action lawsuit, Buckner was experiencing “an open infected wound near her heart from the open-heart surgery, she was on blood thinners, and was receiving antibiotics through an IV.” 

UnitedHealth, in its responses to the plaintiffs, has asserted that federal laws and regulations already require all claims decisions ultimately be made by a human doctor, not an AI program, and raised several other technical legal objections (such as saying the plaintiffs have not fully exhausted their administrative appeal options). The plaintiffs, however, allege that UnitedHealth employees “are instructed not to deviate” from the AI model’s projections, and then tell patients the AI-generated information is proprietary when they or their doctor try to inquire about why their claims are being denied. 

And, to be clear, UnitedHealth is not the only insurer being sued for using AI programs to deny claims. Humana is facing its own lawsuit brought by other Medicare Advantage patients who allege the same nH Predict program also led to their claims being wrongfully denied. Joanne Barrows, then 86, fell in November 2021 and went to a rehab nursing facility after her hospital stay. And just two weeks later, Humana began denying her claims for further care, according to the complaint, leaving her and her doctor “bewildered” because she was under doctor’s orders not to put any weight on her leg.

And the third lawsuit, filed against Cigna, alleges the company used a different AI algorithm that allowed its doctors to deny claims in huge batches — hundreds or thousands at a time — if they did not meet certain predetermined criteria. One patient, Suzanne Kisting-Leung, had claims denied for two ultrasounds done by her doctor to check for ovarian cancer, putting her on the hook for more than $700 in medical bills because, the lawsuit alleges, Cigna’s AI program determined that the scans were not medically necessary. The plaintiffs say that their claims were instantly rejected, without their medical files ever being opened. As for the law that human doctors have to make the final call? According to the lawsuit, the algorithm “enable[s] its doctors to automatically deny payments in batches of hundreds or thousands at a time for treatments that do not match certain pre-set criteria, thereby evading the legally required individual physician review process.”

These people, many of them elderly, had no idea AI was being used to make decisions about their coverage — until it was too late.

“From their perspective, you might as well have been speaking Esperanto. They just had no idea what it was that was being said to them when they were being told. They usually heard about it for the first time when they were being told that they needed to leave,” Glenn Danas, one of the attorneys in the class-action litigation against United, Humana, and Cigna, told me. “They just had no idea what was going on or why this was being done to them.”

AI is now making decisions for traditional Medicare claims too

It’s not just private insurers either. The Trump administration is now pushing traditional Medicare and Medicaid to use AI to adjudicate insurance claims too. In June 2025, nearly two years after those lawsuits had been filed against Cigna, Humana, and UnitedHealth, the US Department of Health and Human Services announced it would deploy a new program called the WISeR (Wasteful and Inappropriate Service Reduction) model to make prior authorization decisions for about a dozen procedures for the people on traditional Medicare in six states — “leveraging enhanced technologies, such as Artificial Intelligence (AI) and Machine Learning (ML), along with human clinical review to ensure timely and appropriate Medicare payment for select items and services.” It would be a pilot program, before federal officials decide whether to take the program nationwide.

It was a quiet, massive change for traditional Medicare, which covers about half of the 66 million Americans on the program. (Medicare Advantage plans, in which private companies administer Medicare benefits under slightly different parameters and which were the subject of the earlier AI-related class-action litigation, cover the rest.) People on traditional Medicare had historically not been required to get prior authorization for most services; now they would be, and a novel AI program would play a significant role in which claims get approved and which ones don’t.

From the start, physician and patient advocates warned about the potentially adverse consequences of the WISeR program; the Center for Medicare Advocacy said the program would likely “complicate, delay and even outright deny necessary care.” According to media reports and internal government documents, that’s exactly what has happened since the model went live in January. 

The Washington Post first reported in March that physicians and their patients were experiencing approval delays or denials for medical services that should have been covered. One glitch led to a denial of payment for services that had been approved. Doctors noticed that certain procedures seemed to give the AI model problems — such as two competing treatments for incontinence — and were weighing whether or not to stop offering one of those services altogether. The Post reported on one Arizona patient who had been receiving steroid injections for chronic back pain before being denied multiple times after the AI program’s launch. The pain became so severe, he said, that he could not exercise at all.

Then this month, the Electronic Frontier Foundation released a trove of documents it received in response to a Freedom of Information Act lawsuit related to the WISeR model. The documents revealed aggressive denials — one company using the AI model had actually denied more prior authorization requests than it had approved — and there were delays even when the care was ultimately approved. One review took 83 days when the model was supposed to make a decision within 72 hours.

The direct feedback from doctors in the federal documents obtained by EFF was devastating.

“We have patients calling our offices crying in pain because their procedures are being delayed while awaiting approvals or guidance tied to this model,” one doctor wrote. “A 3-4 day delay for necessary pain procedures is already difficult for vulnerable patients, but when providers cannot obtain answers for weeks, the situation becomes unacceptable.”

According to the federal documents, the Trump administration was already discussing ways to expand the program. Despite the outcry from doctors and patients, and growing concern among their representatives in Congress, it doesn’t look like WISeR is going away.

How can we make sure AI doesn’t take over insurance claims?

AI is becoming a part of healthcare — it is already happening. And many of those uses could be truly beneficial. Anthropic recently announced an initiative to apply the theoretical findings of its AI models to real-life lab work that could lead to new drugs being developed in the future. In terms of scientific discovery and the potential to translate it into new treatments, the excitement among clinicians about its potential is palpable.

And nobody denies that health insurance administration is chaotic and burdensome and would benefit from being made more efficient and streamlined — tasks which AI is at least superficially well-suited for.

But as the past few years have already shown, rushed and sloppy deployment of AI to take responsibility for approving or denying payment for people’s medical care has created serious problems. The issue is that this technology is being adopted more quickly than any regulator can oversee it. We don’t really know how health insurers are using AI tools. We don’t have a good framework for what keeping a “human in the loop” of insurance decisions — which is what people want — would really mean, Schwarcz said. 

“I think everyone agrees at a high level of generality, ‘Yes, AI should be used to enhance human review and not to replace it,’ but no one actually knows how to write rules to enforce it,” he told me. “I don’t think that there’s anything I’ve seen that has effectively defined what it means for a human to be meaningfully in the loop in the sense of retaining independent judgment as opposed to just reviewing and deferring. It is incredibly hard to specify in rules or regulations a standard that is verifiable and enforceable. That is still, as far as I know, an unsolved problem.”

Some states have started adopting rules requiring transparency for AI use in insurance claims, and lawmakers in Congress have put forward proposals requiring human involvement, including the bipartisan Protecting Patients from Automated Denials Act. But these technologies are so ubiquitous and so flexible that rigorously regulating their use is going to be difficult. Meanwhile, the profit incentive for companies to adopt these tools and deploy them widely is extremely strong. We don’t even know how widespread their use really is.

Danas said he and his clients hoped their litigation would help with “putting the brakes” on AI’s use in insurance. Until we figure out what to do from a policy perspective, what can each of us do in the meantime? Hamer told me that the old advice is still the best advice: If your claim is denied, appeal it. An enormous number of denied claims end up getting overturned on appeal. 

“You do not necessarily have to accept that the first answer from your insurance company is the final determination,” she said.

If there’s one thing that Americans can unite behind, it’s anxieties over artificial intelligence.

According to a recent NBC News/Decision Desk poll, 70 percent of adults say they are personally more worried than excited about the technology; it’s basically underwater in every area of society with one notable exception: its use in scientific research and medicine.

According to Dr. Dhruv Khullar, a physician at Weill Cornell Medicine and writer at the New Yorker, that optimism is well-founded.

Khullar uses AI tools in his own practice as a physician and writes about their advancements in the medical field. He says that “right now, at least, there’s a generally optimistic view of how AI will change healthcare going forward.”

AI in medicine is nothing new. In 2024, the Nobel Prize in Chemistry was given to researchers who developed an AI model that enabled breakthroughs in predicting protein’s complex structures. As it becomes more ubiquitous, it’s also emerging as a resource for doctors to get second opinions and as a tool to take notes on patient-doctor interactions.

On top of that, Khullar says, it might help unlock new medicines to treat diseases. 

Khullar joined Today, Explained co-host Sean Rameswaram to discuss the ways that AI is transforming drug discovery and research, as well as what it still can’t do. They also discuss how to think about AI’s potential in healthcare at the same time as top AI researchers are increasingly warning about the technology’s risk.

Below is an excerpt of the conversation, edited for length and clarity. There’s much more in the full podcast, so listen to Today, Explained wherever you get podcasts, including Apple Podcasts, Pandora, and Spotify.

You’ve been thinking about lifesaving AI innovation. Has AI already started saving lives?

I think it has. This is the fastest I’ve ever seen healthcare take up a new technology. Part of that might be that the healthcare system is so messed up that there’s an appetite for some type of change. It’s unaffordable, it’s inaccessible, it’s inconvenient, the quality is uneven. 

There’s at least a hope that AI is going to help with all of these things, and I think it’s already starting to make its way into the healthcare system. Now it’s our responsibility to figure out how to make the most of it without also promulgating some of the downsides.

For those who are unaware, tell us how it’s working its way into the system.

It’s helpful to categorize it into a few buckets, at least for healthcare. 

The most rapid uptake, and I think the place that it’s already starting to make a difference, is on the administrative side. Many people who have gone to a doctor recently might have noticed that there’s an AI scribe that’s taking notes now. And people seem to really like it, at least so far. Doctors are able to look their patients in the eye in the way that they weren’t able to when they were poring over their computers and just typing what the patient was telling them. 

The second big area is patient navigation. When is your next scan? When’s your next appointment? Do I need to take this medication on an empty stomach or not? There’s a huge opportunity for people, let’s say, who are diagnosed with a serious illness like cancer or heart failure to navigate the system more seamlessly. 

The third big area that I’m really excited about is drug discovery. AI is making a huge dent in the early parts of drug discovery. 

The last thing that I’m excited about and that I use every day when I’m in the hospital is [AI] as a second opinion. Now instead of having to get a consult or turn to a textbook, AI can be a very, very helpful clinical decision support. It’s not to say that I never consult someone, of course, but that first pass — I’m not sure what’s going on. What are some recent trials that might influence my decision here? Is there something I’m missing? Is there a test that I should be ordering that I’m not ordering? — all those types of things, they start to help with.

The biggest chunk of this positive view of AI and medicine is drug discovery. Tell me more about how that’s going so far and how it might be going in a year, or five, or 10.

I don’t want to leave this conversation and say that we’ve solved the cure for cancer because of AI, but I do want people to know that at least for the early stages of drug discovery — figuring out the very basic steps of is this an interesting molecule? Does it have potential biological applications? — AI is already very helpful for that. 

They’re trying to take, basically, what is an infinite number of drug-like molecules that are theoretically possible to be drugs and they’re trying to match that to some disease that’s going on inside a human. And human biology is incredibly complex as well. There’s tens of thousands of genes, hundreds of thousands of proteins, trillions of cells, and you have to match a potential molecule to a potential target within a human. 

In the past, a researcher might take years to study a disease or a biological pathway and figure out, okay, this protein seems to be involved. And actually, it might not be involved, it might be involved, it might be disrupted, but not be the causative agent. There’s a lot of uncertainty there, so now AI can take enormous amounts of data, and it can rank various potential targets that seem to be the most likely to be causing a particular disease and give you that short short list of potential targets that may have taken months or years in the past.

So now you know what to attack, but you have to figure out what to attack it with. So you need a molecule, you need to generate a molecule that’s going to fit into that target or disrupt that target in some way. 

Now again, AI can scour vast chemical spaces and give you a list of the things that seem to be most likely to be able to disrupt that target. And then it can actually help you generate that molecule in some cases. 

So now we have the target. Now we have something that we’re going to attack the target with. And then you have to do something called lead optimization, and that’s the third thing. Lead optimization means you have this lead and you have to optimize it.

“I would make the counterargument that healthcare is ripe for disruption and you don’t even need the frontier frontier models to make it a lot better with AI.”

Now we have to figure out, of all the molecules that seem to be good at potentially affecting this target, which ones are able to be absorbed in the body, which ones are going to get to the right tissue. AI models can predict the most likely molecule to give you that Goldilocks set of properties that’s needed for a safe and effective drug.

As we all know from anyone who’s asked AI to write an email for them or Googled a question, these tools have a certain degree of certitude, even though these tools can just be flat out wrong about stuff. Are you guys worried about that in the field?

Absolutely. And so this is why I think the narrative around AI just replacing scientists or doctors or other workers is incorrect. Because you still need a lot of judgment. 

You need to be able to adjudicate the output of these models to figure out what is most promising and what is potentially dangerous. That is going to require wet labs and scientists and reasoning based on prior experience and understanding the context. There’s all sorts of issues around, can you manufacture some of the drugs that are being proposed by these models? So just because it’s set, it dreams something up doesn’t mean you can actually make that thing in the real world.

Some of the drugs that it proposes might actually be toxic in certain ways that it didn’t predict. And of course, then you have to take this thing into clinical trials. You have to recruit l people who are willing to put this medication in their bodies. You have to find out who might benefit, you have to put it through the regulatory process. 

That’s why I said that the first part of drug discovery, where you’re trying to figure out how do we get the right drug candidates, what seems most exciting to test further, that’s going to be really accelerated. That whole second half where we actually have to figure out if it works in human biology, that is still going to be in some respects an analog process.

It doesn’t sound like you’re terribly scared that we are like ceding control of our hospitals, of our research facilities, to AI. This is very much in the sort of assistant bucket?

For now it is. And I think one of the challenges is to maintain our agency as these models become more and more sophisticated. As you lean more on these machines, if you’re a doctor, inevitably some of the skills, the critical thinking, the reasoning that we put into coming up with the diagnosis that we honed over the course of years, that can start to atrophy. 

These are the types of things that we still need to sort through as we’re implementing more and more AI into the healthcare system.

For the people who just say, shut it all down, this is too dangerous, the risk is far too great, we don’t actually need this, this isn’t doing anything for society, would you make a counterargument?

I would make the counterargument that healthcare is ripe for disruption and you don’t even need the frontier frontier models to make it a lot better with AI.

Even if we wanted to “slow the pace of the frontier,” fine. But there are models from a year or two years ago that could be very helpful in the healthcare setting. When we’re talking about slowing the pace of the frontier and the most sophisticated and potentially dangerous models, fine. 

But if we’re talking about shutting down AI and not using it in biotechnology or not using it in scientific research or not using it in clinical care delivery, that’s where I would push back pretty hard.

Who gets to tell the military “no”?

That question has become even more important after the Pentagon pushed AI companies to loosen limits on how their technology may be used in war — including how much human control must remain over autonomous weapons. When Anthropic said no, it lost a major government deal. OpenAI stepped in, embracing the Pentagon’s more flexible standard instead. Now, thanks to new reporting from The Intercept, we’re getting a better look at what those negotiations actually entailed.

The OpenAI-Pentagon agreement included specific language on the type of AI models the DOD would receive: “OpenAI models that are designed for national security use cases and have minimal refusal rates.” That phrase — “minimal refusal rates” — means that the Pentagon is seeking models that rarely, if ever, tell it no. (According to The Intercept, a Justice Department lawyer representing the Pentagon confirmed this language, before backtracking only a few hours later.)

On the surface, that provision might seem reasonable. The military is a strictly hierarchical institution. When your senior gives you an order, you do it. Leaders generally have more battlefield wisdom and experience than their subordinates, so the rank and file can usually assume that their commanders know what they’re doing and follow along. Plus, in the high-stress, time-sensitive environments of most military operations, there is little room for discussion, explanation, or alternative proposals. 

So, if soldiers can’t refuse an order, why would a machine be allowed to? Who is a robot to tell its commander no?

There’s just one issue. Soldiers can refuse an order — when the order is illegal. In fact, they are legally required to. But by introducing technologies with a limited ability to challenge commands, the military may be making it harder to stop war crimes before they are committed.  

Humans have a duty to disobey

As outlined in US domestic, military, and international law, soldiers are only obligated to follow lawful orders, and they have a duty to refuse unlawful ones. 

The Department of Defense’s Law of War Manual is unambiguous: “Each member of the armed services has a duty to: (1) comply with the law of war in good faith; and (2) refuse to comply with clearly illegal orders to commit violations of the law of war.” Obeying an illegal order — as in a directive that violates the Constitution, US laws, international law, or directs one to commit a crime — can expose a service member, as well as their commander, to criminal liability. 

This “duty to disobey” has foundations in early British and American common law, but it achieved international recognition after the Nuremberg Trials against Nazi soldiers after WWII. 

In those trials, many Nazi soldiers argued they were not liable for crimes they committed during the Holocaust because they were simply following their superiors’ orders. Rudolf Hoess, the commander at the largest Nazi concentration camp, Auschwitz, himself explained: “In Germany it was understood that if something went wrong, then the man who gave the orders was responsible. So I didn’t think that I would ever have to answer for it myself.”

These defenses were unequivocally rejected by the court.

The tribunal held that obedience is not without limits, especially when a wartime act would be clearly illegal to the subordinate or any person of “ordinary sense and understanding.” If “I was just following orders” was an absolute defense, everyone involved in a crime would escape accountability, with the exception of the single highest-ranking officer. Instead, the court determined that individual moral and legal responsibility supersedes national or military chains of command for illegal acts.

Obedience is not without limits, especially when a wartime act would be clearly illegal to the subordinate or any person of “ordinary sense and understanding.”

But, when it comes to agentic AI systems or AI-assisted weapons, increasingly being used in place of human soldiers, does the same logic apply? And, perhaps more importantly, should it?

…But do machines?

When you use an AI chatbot, it does not simply reject queries it may not “agree” with. It does not say no to requests it doesn’t “like.” Instead, it follows a set of established protocols, much as a soldier should follow the law of war: It refuses requests that violate its terms of service. 

The military is using AI for far more consequential applications than a layperson using ChatGPT to, say, build a travel itinerary. So, what kind of world is created by the Pentagon’s request for AI that “minimally refuses” its directives, many of which may bear grave ethical, moral, or legal implications?

Rebecca Crootof, an expert on technology law and professor at the University of Richmond School of Law, told Vox that while the language could easily be read as increasing the likelihood of compliance with unlawful orders, or at least reducing the friction around them, “Minimal refusal doesn’t mean no refusal.”

“I both want to say it’s not definitely going to mean compliance with unlawful orders,” Crootof said, “but also acknowledge that identifying and evaluating what constitutes an unlawful order in the moment can be incredibly difficult.”

Even for human soldiers, disobeying illegal orders is neither simple nor easy. To the contrary, it doesn’t even happen that frequently. In fact, some legal scholars have argued that giving illegal directives in and of itself is an act of abuse against subordinates, who are not thoroughly trained in legal nuances, are conditioned to obey, and face prosecution if they make the wrong choice. 

Still, the mere fact that soldiers can disobey is an important constraint on commanders who might let battlefield conditions, or even commands coming from their own superiors, erode their ethics and commitment to the law.

Still, the mere fact that soldiers can disobey is an important constraint on commanders who might let battlefield conditions, or even commands coming from their own superiors, erode their ethics and commitment to the law.

Those who are optimistic about AI often argue that machines could be better at interpreting and applying the law than human soldiers. Machines can be trained on the law itself — drawing on a repository of legal cases, rulings, and battlefield data — which might make them more “knowledgeable” than an average soldier. And machines do not face the same pressures as soldiers facing prosecution for making mistakes, since they can’t exactly be held accountable; a problem to unpack another time. That, in turn, might make it easier for them to refuse illegal orders in the proper conditions. 

So, is the answer as simple as letting the machines decide? Not exactly. “This has been a desire and a drive by technologists for ages,” Crootof said. “What if we just put the law in the system and have it only do things that are lawful? Isn’t that going to solve things?” 

Interpreting and applying the law, though, is not a simple legal-illegal binary. “It’s very context-specific,” Crootof told me, “and honestly, that’s the kind of thing humans generally find difficult and AI decision-making systems are even worse at.” Often, determining the legality of an act comes down to subtle nuances you can’t program for: You’re allowed to target a combatant; you’re not allowed to target a wounded or surrendering combatant. You’re not allowed to target a civilian; you are allowed to target a civilian directly participating in hostilities. The difference between a legal target and an illegal one may come down to body language, clothing choices, or even battlefield intuition. 

As Crootof explained, “These are not the kinds of contextual analyses AI is good for.”

So what’s the solution?

Where human soldiers’ power to disobey would traditionally serve as an important check on the power of military commanders, if an imperfect one, replacing soldiers with machines that don’t have such safeguards removes one last layer of judgment over the use of force. And yet, machines themselves are not always capable of making the judgment calls that go into interpreting the law, so it may not be appropriate to let AI have the final say either.

So what’s the solution? Given the way AI technology has already been thoroughly integrated into the military, Crootof told me it may be somewhere in the middle. 

Rather than programming AI to refuse commands it interprets as illegal, Crootof said, “what it could be designed to do is flag those indeterminate zones, those gray-zone analyses, for human review or for a higher authority.” When a system picks up conflicting signals — an enemy soldier who looks like he may be surrendering, for example — it could prompt the commander to confirm their selection or remind them of applicable law. 

In this way, a check on commanders’ battlefield authority could be preserved without fully ceding judgment over the use of force to the machine. 

We don’t know yet whether OpenAI has equipped its “minimal refusal” models with such capabilities. But as those technologies become increasingly central to military decision-making, the public deserves answers sooner rather than later.