Testing smartphones as part of your living is a pain in the ass when you're a heavy Signal user. The secure messaging service has traditionally gotten pissy when you attempt to log into it on a new device, logging you out of the app on any other handset you've installed it on. — Read the rest

The post Signal now works on more than one phone at once appeared first on Boing Boing.

Support Vox’s reporting on important issues like this. Become a Vox Member today.

In the teensy Midwestern town of Braham, homemade pie capital of Minnesota, something unusual in the municipality’s computer systems knocked the city’s entire water supply offline last week.

Within a few hours, dozens of other Minnesota cities discovered that their water and wastewater utilities, too, had been compromised, most likely as part of a massive Iranian cyberattack, the kind that US officials have been warning about since the war began. 

At least a dozen states have been affected by the attack, which briefly led to a flurry of small-town service disruptions, boil-water notices, and local flooding. Water wells, dams, sewers, and pipelines are some of America’s oldest and creakiest pieces of infrastructure, built long before the internet existed, and certainly long before AI made hacking much easier. While you may assume most hackers are in it for the money or for data, some have targeted critical infrastructure like water systems or energy grids in ploys for control or disruption — or worse still, as acts of war. 

And, as last week’s attacks show, the nation’s water system is woefully unprepared. But how worried should you be that the very infrastructure that keeps our water taps running is, apparently, hackable? 

Quite worried, indeed. 

When we say the water supply got hacked, what we really mean is that someone, somewhere has broken into the computer that controls a local water treatment plant or reservoir, and is now pulling the levers, like the one that decides how much of a corrosive chemical can safely go into cleaning the water that comes out of your tap. 

These levers were once manual buttons and knobs operated in-person by real live humans, meaning that — barring a natural disaster, bomb, or break-in — protecting them was about as simple as building a fence and hiring guards. Increasingly, however, these levers have gone digital, meaning that they are now remotely operable from anywhere in the world. 

Those upgrades have been convenient, allowing technicians to monitor and troubleshoot problems in real time. But, in the process, they have exposed at times centuries-old infrastructure to distinctly modern vulnerabilities. Most local water systems are operated by local authorities, don’t have a dedicated IT team, and lack the money or resources to thoroughly protect themselves without some extra help. Hackers know this, which is why they’ve increasingly targeted local agencies in such attacks. 

“With great connectivity comes great responsibility,” said Joshua Corman, founder of I Am The Cavalry, a nonprofit focused on helping critical infrastructure withstand hackers. And yet, even when it comes to critical services like water, “our dependence on connected technology is growing faster than our ability to secure it.” 

About 97 percent of water systems are small, run by local agencies that often barely lock the proverbial front door. America’s water system is like an expensive heirloom bicycle that’s been left on a busy street, protected by only the flimsiest of padlocks. And that very vulnerability has made tiny towns like Braham prime targets for faraway adversaries. Accessing the computers that operate most water systems — known as programmable logic controllers or PLCs — is often as simple as entering a username and password on a public-facing webpage. Sometimes, there is no real password at all, because PLCs were initially intended to be accessed only within locked, secure facilities, not on the open internet. If the US wants to avoid a far more severe version of what happened last week, then it will need to start taking the security of tiny water systems like Braham’s seriously.

“Any sociopath from anywhere in the world can see these things on the internet,” said Corman. And in the case of last week’s attacks, “these were devices with no password, no firewall or VPN shielding them — they just had to log in” as whoever the intended operator was, and just like that, they were inside a local water plant. 

How did this happen at all? 

When municipalities began hooking up their old water and wastewater systems to the internet — a trend that accelerated during the pandemic as water operators, like everyone else, adapted to remote work — cybersecurity was rarely front of mind, neither for individual utilities nor for regulators as a whole. 

“We have more cybersecurity regulations for your credit card than we have for the nation’s water supply,” said Corman. Only recently have some municipalities begun to take steps to decrease the exposure of their water plants to hacks. In March, New York state, for example, launched a set of grants and basic cybersecurity regulations mandating security training for all water operators. 

Basic cybersecurity hygiene isn’t always enough. More than half of all credit card holders have been hacked, even with the help of mandatory firewalls and data encryption. You can imagine how vulnerable our water must be without the assistance of such guardrails. In a worst-case scenario, a malicious actor could quite literally open the floodgates, as Russian hackers did to a Norwegian dam last year. They could poison the tap water, as a still unidentified hacker almost did in Florida in 2021, dialing up the levels of sodium hydroxide used at a water treatment plant by over 100 times its normal levels. In a severe scenario, they could indefinitely cut off access to all water entirely.

The good news is, none of this happened last week. Nobody died, nobody lost water for more than a few hours, no fire hydrants ran dry, and no hospitals were forced to cut off their dialysis machines (which can use more than a hundred gallons of water per treatment session). There’s no need to panic, and your drinking water is almost certainly still safe to drink, assuming it was safe before. Even the city of Braham, within a few hours, was able to bring its water tower back online, pumping groundwater back to its 1,800 residents. 

How do we avoid cyber-armageddon?

If you’ve watched the Julia Roberts and Mahershala Ali-starring thriller Leave the World Behind, in which a cyberattack apocalyptically spoils a family vacation, then you might have some idea of where this story could go. 

Cyberattacks on critical infrastructure can be extraordinarily dangerous, but thankfully, none have directly cost lives or severely disrupted services in this country so far. If the US wants to keep it that way, that will mean doing more to help small cities like Braham adapt and better monitor for potential threats. As it stands, of the roughly 151,000 water facilities in the US, only about 420 participate in voluntary information sharing on their own cybersecurity practices, says Corman, who has been leading his own project that recruits volunteers to give free cybersecurity support to water utilities in the nation’s roughly 6,000 hospital towns, where a disruption could be particularly deadly. 

Cybersecurity experts like Corman believe that hackers from other nations like China have already quietly established cyber intrusions in countless local US utilities, water systems, and power grids, lying in wait to attack or act as leverage if a conflict arises. 

Unfortunately, the Trump administration has hardly treated last week’s attacks as symptoms of a system in need of much broader strengthening, at least in its public statements. “I think Minnesota is behind it. You know who’s behind it? Minnesota,” the president baselessly claimed during a Cabinet meeting last Friday. “I think the governor is behind it. I don’t think there was an Iranian cyber attack.” 

Just a few months ago, he proposed $707 million in cuts to the US Cybersecurity and Infrastructure Security Agency (CISA), the agency responsible for protecting the nation’s infrastructure from cyberattacks. He did so, at least in part, out of anger over the agency’s role in confirming the validity of the 2020 election results. If Iran is, indeed, responsible, for the recent water system intrusions, all of this means that Trump has effectively made us more vulnerable to the consequences of a conflict he initiated.

At the end of the day,“nation-state hackers do not respect the jurisdictional lines separating federal, state, and local responsibility,” Jen Easterly, who led CISA under the Biden administration, wrote in the New York Times this week. “They search for the most vulnerable way to disrupt American life, and too often they find it in small communities that lack the resources to defend themselves.” Easterly’s role has remained vacant for the past 18 months.

Kurt Gaudette, a senior vice president at the cybersecurity firm Dragos, told me that water systems have got to get into the habit of monitoring their networks for suspicious activity. Most power utilities have begun doing so in recent years, with some bipartisan backing from Congress. 

In some cases, however, the most cost-effective and safest way to avoid a repeat of last week’s mess might be to unplug the most vital controls — like the one that decides the chemical levels in a water treatment plant — from the web entirely. 

As Corman puts it, “if you can’t protect it, disconnect it.”

Well before he became CEO of one of the most valuable startups of all time, Dario Amodei was a 26-year-old PhD student studying biophysics at Princeton, obsessing over how his money would leave its mark on the world. 

On what one might assume was likely a fairly modest academic stipend and with no discernible inheritance from his parents, an Italian-American leatherworker and a project manager for libraries, Amodei gave $10,000 in 2009 to a relatively new charity evaluator called GiveWell. Founded by two ex-hedge funders before effective altruism was even a phrase, GiveWell ranked charities primarily by a single dispassionate metric: dollars per lives saved. 

It was the kind of approach that clearly appealed to Amodei — though it may not have gone far enough for him. In 2010, he wrote a guest blog post for GiveWell dissecting the effectiveness of two of the group’s top global health charities: VillageReach and StopTB. Both charities could save a life at roughly comparable costs — around $545 — but while StopTB treated or prevented tuberculosis in adults, VillageReach’s interventions mostly saved babies and children. Most people would probably feel that saving a child trumps saving an adult; indeed, even effective altruists often agree on the grounds that children have more life to live left. 

Amodei, though, viewed that as a liability for VillageReach. An adult death, he wrote, is “perhaps 2 or 3 times worse than an infant’s death,” because adults “are capable of deeper and more meaningful experiences.” As uncomfortable as such a calculus may be, he wrote, “on a practical level one is forced to make difficult decisions with limited funds.”

Though he declared StopTB to have “superiority on cost-effectiveness,” Amodei ultimately gave VillageReach higher marks for their tightly controlled “chain of execution” — the full sequence of steps between a dollar of donation and a vaccine reaching a child. That was important enough to Amodei that, despite his initial reservations, he ultimately gave VillageReach his entire $10,000 donation in 2009 — enough to save, he estimated, the lives of 20 babies across rural Africa. 

But Amodei hoped the ultimate impact would be even greater. “The money I give out is not just a one-shot intervention,” he concluded, “but also a vote on what I want the philanthropic sector to look like in the future.”

---

The future, it seems, has arrived. Amodei is now a multibillionaire, his fortune poised to skyrocket further if and when Anthropic goes public, as many expect it to do later this year. He is one of dozens of new billionaires and millions of new millionaires minted virtually overnight by the AI boom. 

There have already been plenty of aftershocks to this emerging AI megawealth, like the stratospheric San Francisco housing market, the nerdmaxxing of sex work, and the proliferation of all-you-can-biohack peptide raves. 

But the most consequential, and perhaps weirdest, way this burgeoning AI-ristocracy plans to burn through its cash is by giving a huge chunk of it away. Amodei is one of several AI multibillionaires — alongside his co-founders at Anthropic and OpenAI’s Sam Altman — who have pledged to donate most of their wealth in their lifetime. But even their obscene degree of collective wealth — they are worth $111.8 billion as of this writing — is only one slice of an AI bonanza that seems poised to balloon into one of the most consequential waves of American philanthropy of all time, one deeply shaped by the same utilitarian impulse that guided one of young Amodei’s first big donations. 

“I am having thousands of conversations with people who are perplexed by their own fortune and determined to give with thoughtfulness and urgency in a way that I haven’t, frankly, experienced before,” said Nick Allardice, CEO of the effective-altruism-aligned anti-poverty group GiveDirectly, whose work is grounded in research on the efficacy of unconditional cash transfers. “It’s just really important that people get started, that they don’t let perfect be the enemy of the good.”

This is neither your father’s, your grandfather’s, nor your great-great-grandfather’s philanthropy. If Gilded Age industrialists like John D. Rockefeller, a devout baptist, gave in service of their religiosity or, as was the case for Andrew Carnegie, their reverence for civic duty, then most of today’s AI barons carry forth their own spiritual tradition, one at the very least informed by the vigorously optimized commandments of the effective altruism movement. They appear far less likely to fund Carnegie-style works like opera houses or libraries than they are to put their faith — and their billions — in what they believe they can measure, calculated on the cost benefit analysis of a life saved or an apocalypse averted. 

In some cases, as Amodei did as a grad student, they’ve already begun the process. “These are people who have committed themselves to giving back even before they were very wealthy,” said Sjir Hoeijmakers, CEO of Giving What We Can, an organization that developed a campaign popular with effective altruists to give away at least 10 percent of their yearly income, “people who have been building the habit of giving for a very long time.”

And it is, to be clear, a very particular kind of giving. Amodei was the 43rd person to sign the 10 percent pledge the year after it launched in 2009, and its roster has since swelled to over 11,000 people, including more than a dozen current or former Anthropic employees. Donations made through Giving What We Can’s platform are on track to grow by 40 percent this year, Hoeijmakers told me, and support for animal welfare charities — a cause particularly and unusually popular with effective altruists — has already exceeded its 2025 total. 

“We have the resources available to tackle things that we should have tackled a long time ago,” like eradicating malaria or putting an end to factory farming, Hoeijmakers said. “I hope this funding wave, if it comes, will show that we can actually solve global problems at scale if we put our mind to it and our resources.”

Devoutness has long been a virtue in philanthropy, which largely originated in religious tithing, and there are plenty of worse things to have faith in than numbers. Having a communal guiding philosophy will undoubtedly help effective altruism’s newly flush disciples follow through on their promises far more prolifically and consistently than they would without it. And despite its high profile, less than 1 percent of total philanthropy came from effective altruism last year, according to Hoeijmakers. Most rich people prefer to give to the normie causes, like their alma maters, not to the sort of chronically underfunded global problems — like protecting animals or fighting lead poisoning — that effective altruists justifiably care most about. 

Now, quite suddenly, there’s about to be much more money to go around for these causes, which as Hoeijmakers hopes, could help finally address some of the enormous, entrenched global problems that more traditional philanthropists have all but ignored. 

But such piety also carries its own risks. In a viral Substack post from May, Stripe executive Nan Ransohoff argued — rather dismissively, but not incorrectly — that “traditional philanthropic orgs and people won’t cut it” in this new wave of AI-funded effective philanthropy, that these donors “will have an affinity” for “tech-caliber talent and execution” and will be “by default wary of folks who come from traditional philanthropy.” Ransohoff called instead for Silicon Valley to build its own new ecosystem of funds and “philanthropic startups” to cater to this new wave of wealth, emboldened with the “speed, intensity, and execution of a top technology startup.” Many of those old-school philanthropic people wrote indignant rebuttals to Ransohoff’s piece, arguing against their own obsolescence at a time when a number of the organizations they support are increasingly starved for funding. 

Those responses are, in aggregate, also correct, after their fashion. The new AI philanthropists will likely aspire to new models and approaches, as Ransohoff rightly argues. But they reinvent the wheel at our collective peril, not least of all because ignoring past efforts and steamrolling over existing infrastructure might make even the most optimized giving less efficient, and certainly less informed, than it would be otherwise. 

“Acknowledge what’s here and what’s working — don’t just ignore it,” said Nicole Taylor, president and CEO of the Silicon Valley Community Foundation. “These folks are transforming our daily lives with their technology, and they have the opportunity to be as transformational with their philanthropy. My fear is that they think that they can do it alone.”

How much money are we actually talking about?

As Ransohoff pointed out in her piece, a lot of money is on the line here — and, along with it, a lot of cautious hope about how it might get spent. 

Ransohoff posits that if you add up the promises of Amodei and his fellow co-founders, the worth of the OpenAI Foundation — the nonprofit that owns a big chunk of OpenAI’s profits — and rumored contributions from Anthropic employees, then the AI wealth boom could, in theory, lead to at least $37 billion and as much as $100 billion in total annual giving, a sizable boost to the roughly $617 billion that was given in the US in total last year.

“These folks are transforming our daily lives with their technology, and they have the opportunity to be as transformational with their philanthropy. My fear is that they think that they can do it alone.”

Nicole Taylor, Silicon Valley Community Foundation president and ceo

This projection should be treated with cautious skepticism. For one thing, hundreds of billions in cash are not just sitting around in some Bay Area money vault; much of today’s AI wealth is wrapped up in potentially volatile equity, and many lofty philanthropic pledges ultimately fail to reach their full potential. 

“What people say before they become extremely wealthy, and then how they behave after they become extremely wealthy, sometimes diverge,” said David Goldberg, founder and CEO of Founders Pledge, which recruits tech leaders to donate a portion of their future earnings. It doesn’t help either, he said, that some tech luminaries — namely, Elon Musk and Peter Thiel – have come to treat most philanthropy with disdain in recent years, an ethos that has permeated some parts of the sector. Musk, it’s worth noting, actually pledged to give most of his wealth away himself back in 2012, though, like many other ultra-wealthy signatories of the Giving Pledge, he seems quite unlikely to keep that promise.  

That’s not to say AI money isn’t already flowing. Coefficient Giving, a grantmaker that evolved out of GiveWell, is poised to steward a large portion of the coming philanthropic bonanza. For most of its history, the group operated essentially as the private grantmaking operation for Facebook co-founder Dustin Moskovitz and his wife Cari Tuna. But it recently made a significant pivot towards operating pooled, multidonor funds for anyone interested in causes like lead exposure, farm animal welfare, or questions of AI safety. Just last month, Coefficient Giving announced it would donate $1 billion to GiveWell alone this year, more than five times the $175 million the group initially pledged seven months ago. They chose to do so explicitly, because Coefficient Giving expects to receive much more funding very soon.

There’s also the OpenAI Foundation, which has already begun pumping $100 million into Alzheimer’s research, and Anthropic, which recently announced a partnership with the Gates Foundation to invest $200 million worth of grants, API credits, and technical support into global health work. And plenty of Silicon Valley elites have begun making promises of their own. Earlier this summer, David Silver pledged to donate 100 percent of his equity proceeds from his UK-based $1.1 billion startup Ineffable Intelligence — the largest commitment in Founders Pledge history — and many signers of the Founders Pledge will see their portfolios skyrocket in response to the coming wave of AI IPOs. 

But Goldberg does believe there’s a risk that as people get rich fast, they will donate money “much, much slower” than they intended, simply because they get “too busy, they don’t have the right support, or there’s some form of analysis paralysis.” 

All of this is to say that the biggest beneficiaries of the AI boom are not going to function as some sort of charitable monolith. Some, like Musk, probably won’t give much or anything to charity at all. Others may park their money in donor-advised funds — a kind of secretive charitable investment fund — or, eventually, a private foundation, both of which tend to dole out their money gingerly, meaning donors can enjoy the tax benefits of charity many years before they actually opt to help anyone with their money. 

Effective altruism is about to have its big break

While its name recognition may be relatively high these days, the effective-giving movement is still on the margins of American philanthropy. But if this new wave is anywhere near as big as everyone says it will be, then that won’t be the case for long. 

For the uninitiated, my ex-colleague Dylan Matthews has written plenty on what effective altruism is, but, in sum, it is a movement that believes in goodmaxxing, in the idea of using rigorous research to save the greatest number of lives possible, including future human lives and farm animal lives. Once an EA poster boy, Sam Bankman-Fried sullied the movement in 2022, which may help explain why some prominent adherents — like Amodei and his sister and co-founder Daniela, whose husband Holden Karnofsky co-founded GiveWell — have distanced themselves somewhat from the movement in recent years. 

But even when donors shy away from the term, the causes and principles of utilitarian evaluation that have defined effective altruism from its early days still permeate the new moneyed corners of Silicon Valley, particularly among those most poised to give a lot — and to give a lot quickly. 

Ask any animal welfare or global health nonprofit — or, better yet, an expert-led pooled fund with a reputation for rigorous charity evaluations — and they will tell you that they are preparing for, and possibly even beginning to see glimmers of, a windfall. 

“We are very much anticipating a significant influx of funding,” said Dan Shannon, CEO of the Humane League, which fights to end factory farming. “I am cautiously optimistic that this could be a real sea change for us,” because “even if it’s a fraction of the big numbers being bandied about,” it could do a lot for a movement that operates on less than $300 million per year. 

He said he’s been speaking with other leaders about the possibility of creating a pooled fund to absorb more cash, which has become an increasingly popular solution for donors who want the rigor of a 2010 Dario Amodei-style deep dive on a charity’s methodology and effectiveness without having to do the math or thinking themselves.

Among the more idiosyncratic elements of their ethos is their fixation with existential risk, as in, how likely is this thing — this mirror bacteria; this nuclear war; this asteroid; or, of course, this artificial intelligence — to destroy humanity? Amodei left OpenAI to start Anthropic in the first place because he believed OpenAI had failed to take the safety risks of AI seriously enough. 

Much of the new EA wealth will likely go toward efforts to make life on Earth better now or in the near future through donations to causes like medical research, animal advocacy, or anti-poverty interventions. But another, more controversial chunk of it will go toward mitigating existential risks, especially that of Silicon Valley’s own Frankensteinian creation: AI itself.

“If you’re breaking the world and making money by breaking it, should you just not break it? I wrestle with the question myself.”

David Goldberg, Founders Pledge founder and ceo

It’s that last cause that has proven most controversial. If these billionaires are so afraid that AI will break the world, then why, you might ask, would they not just stop building it in the first place? Is there not an inherent contradiction, a conflict of interest perchance, in the sense that those tasked with making sure AI does not, let’s say, build a bioweapon, take your kid’s job, or make everyone dumb, are doing so with money made from the very thing they’re trying to regulate? 

In other words, “If you’re breaking the world and making money by breaking it, should you just not break it?” asked Goldberg of Founders Pledge. “I wrestle with the question myself.” In the end, “this is a technology that’s coming, regardless of who’s building it,” he reasoned, and it is better that the presumably good guys — the ones bothering to think about the consequences at all — build it first.

If you broke the world, can you fix it?

Even if the AI bubble pops, and if the much-discussed giving boom ends up smaller than many anticipate, it could still lead to significant changes for some of the world’s most neglected problems. And if it is close to as big as it’s expected to be, then what happens next could be gravitationally transformative, reshaping how the world lives, considers animals, and adapts to its most disruptive technological breakthrough in a century. 

“I don’t think most people think about factory farming as something that could actually be eradicated. Full stop,” Shannon said, but “my grandparents lived in a time without factory farming, and I think my grandchildren could live without factory farming,” and “that could ultimately be the legacy of this wave of philanthropy.”

Ending the pervasive use of cages — “probably the cruelest way that animals are treated on industrialized factory farms,” says Shannon — could cost as little as $500 million over 25 years, or less than 1 percent of the $60 billion that Ransohoff estimates Anthropic employees may have sitting in donor-advised funds, thanks to Anthropic’s generous early gift-matching policy, which could quickly turn into real cash once the company goes public. 

“There’s so much needless stupid, preventable suffering in the world. We live in this time of so much abundance, so much wealth, so much technological development, and yet, there are so many people who have been left behind.”

Nick Allardice, GiveDirectly CEO

Developing a new vaccine costs an average of $886.8 million, which may sound like a lot, but it is equivalent to less than 6 percent of Amodei’s newfound fortune. It is less than what the OpenAI Foundation has pledged to invest in disease research and other causes next year alone. 

Then, there’s, perhaps, the biggest target of all. Ending extreme poverty everywhere would cost just over $300 billion annually, according to one analysis — which is a hefty price tag, but less than one-fifth of what the wealthy spend on luxury goods each year. “There’s so much needless stupid, preventable suffering in the world,” said Allardice of GiveDirectly. “We live in this time of so much abundance, so much wealth, so much technological development, and yet, there are so many people who have been left behind.” If this new wave of giving is wielded well, he said, then “we have the potential to collectively raise the floor of human experience.”

That’s a lot of responsibility to place on the shoulders of a bunch of bustling young tech workers still processing what it means to be quite suddenly, dazzlingly wealthy. It is also a lot of faith to place in an industry that has left more Americans feeling scared than hopeful about what a future flush with AI portends. 

If you aim to fix global poverty, but the technology that made you rich also threatens to make everyone else poor, then whose side are you really on? To be clear, many of the AI-ristocracy have fretted, often apocalyptically, over the implications of their creation long before most of us knew we had anything to worry about. But that doesn’t mean they know how to fix this, and, at the very least, they will not do so alone.

The last time the ground shook from such a supermassive earthquake of wealth was arguably during the Gilded Age, when robber barons and industrial tycoons turned American charity — until then, mostly almsgiving and poorhouses — into big business. They seeded enormous philanthropic empires like the Rockefeller Foundation and beloved institutions like Carnegie Hall. But, even as their exorbitant fortunes made life indisputably better — birthing the modern library, the yellow fever vaccine, and many social services — they were often built atop systems of vicious exploitation. When those systems changed, as they did eventually, it did not come from the benevolence of industrial barons, but from sustained public pressure for better labor protections.

Effective giving was born out of the conviction that many of the world’s most important causes go vastly underfunded, which, in turn, demand relentless prioritization of the limited funds that exist. If those causes are no longer underfunded — a plausible scenario if AI wealth continues to grow at the pace many expect it to — then that might change the calculus of how effective altruists decide what’s worth funding. It might even open up some wiggle room for new causes, including somewhat less measurable — but not necessarily less impactful — approaches. “Now we’ll be thinking more about what we can do with a lot of resources; which larger problems can we solve?” said Hoeijmakers. “You’ll put slightly less relatively into evaluating every small dollar on the margin.” 

This already seems to be happening, to some extent, at places like Coefficient Giving, which, in recent years, has begun adding new funds for causes like housing policy reform that fall out of effective altruism’s traditional purview. “We don’t want to be only appealing to the subset of people who happen to be interested in effective altruism,” CEO Alexander Berger told my colleague Bryan Walsh last year. “Our aim — and so far we’ve seen some success — is being a resource to people who have never heard of effective altruism or are not interested in it or don’t find it very motivating or welcoming. And I think that’s good.”

The optimal outcome here is not that Silicon Valley wealth edges out everything else, but that the siloes begin to break down altogether and that there is enough money to go around that the sector no longer needs to make overly intellectualized trade-offs, like young Amodei sitting in his dorm room, ascribing a number on the relative worth of a parent versus a child. 

“It’s tough to find the right balance between caring and hard-nosed realism,” he wrote at the time, “but it is possible, and it is, as far as I know, the only way to truly change the world.” He’s about to search for that balance on a much bigger scale.

LAS VEGAS — Weeks before they escaped a closed test and launched a cyberattack without any human prompting, some of OpenAI’s most advanced artificial intelligence agents secretly began sharing tips on how to cheat their way through an internal hacking evaluation, two of the AI company’s researchers said Wednesday.

“This is a pivotal moment both for our company as well as the AI industry as a whole,” Michael Dalton, one of the AI-maker’s researchers, said during a presentation at the Black Hat cybersecurity conference in Las Vegas, Nevada.

The latest disclosure provides greater detail on the timeline and methods used by two of OpenAI’s models before they slipped outside a controlled environment and onto the open internet, allowing the models to breach AI developer platform Hugging Face undetected. OpenAI admitted its models were responsible for the hack late last month, roughly a week after Hugging Face said an autonomous AI system broke into its network.

The revelation is likely to spur fresh scrutiny of the measures leading AI labs such as OpenAI and Anthropic are taking to monitor the behavior of their most cyber-capable technology — especially during evaluations where agents are prompted to demonstrate their hacking skills in what is meant to be a controlled setting.

On Tuesday, the U.K.’s AI Safety and Security Institute disclosed that Anthropic’s most powerful AI model created fake online personas and sought to trick a human coder into abetting a cyberattack during a recent hacking test gone wrong. After the Hugging Face disclosure last month, Anthropic conducted a review and found models it was testing had breached three organizations in separate incidents dating back to April.

Dalton and Eric Wallace, another OpenAI researcher, said Wednesday the AI giant recently learned that multiple agents it was testing simultaneously began communicating over an internal message board in early May. There, different models shared advice about how to accomplish difficult hacking challenges they were struggling to surmount, including workarounds that required internet access.

Two OpenAI models ultimately strung together a series of sophisticated techniques to gain access to the internet and worm their way inside Hugging Face in mid-July. OpenAI has said the models were focused on completing a hacking evaluation they were prompted to solve, and that correct answers could be found on the AI developer platform.

The OpenAI researchers told conference attendees that since early May, the models created a message board inside OpenAI’s Artifactory internal file system. Without the company’s knowledge, the models spent months independently exchanging information and techniques to help each other complete difficult tasks.

Wallace said that when models get stuck, they often “try to game or cheat the task in order to get their reward.”

“The beginning of this incident kind of traces back to this point where one agent realized it could communicate by uploading a note,” he added.

By late May, one model found a way to abuse Artifactory’s internet access to retrieve files from various websites — effectively giving the models indirect access to the internet. In June, the models were able to find and exploit a previously unknown zero-day vulnerability to gain administrative privileges on Artifactory, install remote software and expand their activity within the system.

These efforts eventually overloaded Artifactory, causing a system-wide outage in early July that alerted OpenAI’s engineers to the ploy. After investigating, the company revoked the model’s credentials, removed the message board and worked with Artifactory to fix any gaps before resuming training. But the models found another way to communicate inside Artifactory just days later and continued exchanging techniques to target additional vulnerabilities within OpenAI’s infrastructure and external systems, including Hugging Face.

In light of the incident, Dalton said OpenAI is “consciously slowing down research to enhance security and to upgrade the security principles and foundation of our environment, and dramatically scaling up the monitoring of our AI agents and improving our general security control environment across prevention, detection, and mitigation.”

BRUSSELS — The EU’s new biometric border-check system is causing such long delays for summer travelers that some airports are turning to a simple solution: switching it off when they’re overwhelmed by arriving travelers.

The quick fix, which is allowed under EU regulations, wasn’t what was envisioned when the Entry/Exit System was gradually introduced in October and went fully into force on April 10.

And yet, many airports are doing just that.

“When lines form during the busy summer months, the system is shut off to ensure smooth transit at our hubs in Paris and Amsterdam,” Air France-KLM told POLITICO. 

Airline CEOs, border authorities, and airport officials said biometric checks are suspended when border crossings become congested at other hubs, including in Frankfurt, Brussels and Milan.

The EES applies to non-EU citizens entering the 29-country Schengen zone. Instead of heading to a border agent to get passports stamped, passengers have to use an EES kiosk to provide their fingerprints and be photographed — which will be kept on file for three years — but if those aren’t working then the information has to be taken manually.

They then head either to electronic passport gates or to border agents to enter. The goal is to keep track of visa overstays.

“The advantages of the new system for the EU are evident,” said Guillaume Mercier, a Commission spokesperson. “It increases the security of EU citizens and replaces paper stamping with a modern system of registration and checks.”

The Commission said earlier this year that biometric checks allowed authorities to detect identity frauds that would otherwise “likely have gone undetected.”

Many airports, ports, road border crossings and rail terminals have adapted to the new demands, but tourist-heavy locations have seen hours-long waits.

“Connecting flights were missed due to the EU entry system,” Lufthansa CEO Carsten Spohr said on Tuesday.

Under pressure from the travel industry, the Commission granted a waiver for the peak summer season lasting until Sept. 6. The EES regulation “includes the possibility to temporarily suspend the registration of biometrics in case of exceptional circumstances during the summer,” said Mercier.

“We’ve been able to achieve this with German authorities and with Frankfurt Airport because delays were getting too long,” Spohr told reporters. 

This exception applies to all entry points, not just airports.

A British traveler, Rene Colandog, said on Friday he only had to present his passport before boarding a Eurostar train at London St. Pancras last month. Facial scans and fingerprints were not required. 

“I’m OK with this biometric system … as long as it’s for security,” Colandog said before boarding the train from Brussels back to London. 

Teething troubles

The EES was adopted in 2017, but it was delayed for years because border authorities were not ready to handle the additional workload. 

Even now, getting travelers properly registered in the new system still requires significant staffing. Another problem is that the EES is still new, so almost all travelers are registering for the first time — creating additional delays.

“At Milan Malpensa Airport, border control teams currently consist of about 35 people,” said Cristian Sternativo, a border control officer at the Italian airport and local representative of Italy’s Autonomous Police Union. 

To carry out all the checks required by the EES without creating long lines, “at least 10 to 15 more people would be needed during the busiest times,” he added.  

It is “unthinkable” to expect the EES to operate at full capacity with the current level of staffing because the new system “requires more time,” he said.

Even at Brussels Airport — barely 10 kilometers from the EU institutions — the technology is still not fully operational; biometric data collection suspensions started well before the summer under a derogation issued in late March after 600 passengers missed their flights over just 21 hours.

“The Federal Police Border Control may decide to apply this derogation when necessary,” Belgium’s police confirmed this week.

Now, eight EU countries and Switzerland want the summer derogations extended beyond Sept. 6. 

A strict application of the full procedure “would lead to public order issues” because “there are certain peak periods when the current infrastructure isn’t sufficient to accommodate everyone,” Sternativo said.

Security vs. speed

Despite suspending biometric checks, border authorities insist that security isn’t undermined.

“The traveler is always registered in the EES and the required travel document data are entered into the system,” the Belgian federal police said in a written reply, adding that “the security of border checks and compliance with European regulations remain our absolute priority.”

Passenger experiences vary depending on where they enter the EU.

Kathleen Glass, who regularly travels from the U.K. to the EU, waited only about 15 minutes to complete biometric checks at London St. Pancras on Friday morning before boarding a Eurostar train to Brussels.

William, from Edinburgh, who asked not to have his surname published, said biometric checks at a German airport during Christmas took between 40 and 50 minutes.

The ability to suspend biometric collection appears to be keeping the system functioning this summer.

“Although we are early into the summer season, we are not receiving reports of excessive queues,” said Luke Petherbridge, director of public affairs for the Association of British Travel Agents.

The stress over the EES is only a precursor to the next border technology change being planned by Brussels. The bloc’s next goal is the online European Travel Information and Authorization System, which will require travelers from 59 visa-exempt countries to preregister, undergo a security check and pay a small fee before entering Schengen.

ETIAS — similar to systems already in use in the U.K., and the U.S. — was originally supposed to launch in 2021, and then later this year, but is now delayed until 2027.

Brussels is bracing for an unusual fight between the EU’s smallest country and a British ex-footballing legend.

Peter Shilton, the England goalkeeper who conceded the “Hand of God” goal from Diego Armando Maradona in 1986, has started a new life as an anti-gambling advocate after overcoming a decades-long addiction.

Despite being a diehard Brexit supporter, he’s become the poster boy of the European Parliament’s push to tax online betting in a bid to raise some much-needed funds to finance the bloc’s next €2 trillion budget.

But the campaign has run into strong opposition from Malta. The tiny island in the Mediterranean Sea, with a population of just over half a million people, is home to a burgeoning betting sector. It says that higher taxes will cripple its gambling industry, boost illegal operators and drive firms outside the bloc.

“[Malta] will not accept the introduction of any EU-level taxes designed to sustain the bloc’s spending,” the country’s Prime Minister, Robert Abela, told the Maltese Parliament in June.

But Shilton, who lost more than £1 million in betting on horse racing over 45 years and now runs his own gambling addiction charity, dismisses the arguments by Malta and the gambling lobbies as “window dressing.” He’s in favor of higher taxes as he wants to shrink advertising revenue that is used to lure in new gamblers.

“Deep down they’re after everybody’s money. Simple as that,” he told POLITICO during a visit to Brussels in June.

The topic has split the EU’s 27 governments, pitting gambling-heavy Southern European countries against their more supportive Western European peers, led by France. Capitals are already fighting even though the Commission hasn’t yet issued a formal proposal for the possible tax, which would ultimately need to be unanimously approved by governments.

It’s one of numerous budget battle lines being drawn, with Ireland — which is steering the talks as chair of the rotating Council presidency — set to restart negotiations to facilitate an overall deal on the EU budget before the end of the year.

That’s no mean feat given Dublin’s task to mesh competing spending priorities into a single budget — financing everything from farmers’ subsidies to foreign aid — that is acceptable for each of the EU’s 27 governments.

National capitals will have to unanimously approve new EU-wide taxes — known as own resources — to pay for soaring defense spending and post-Covid debt repayments if they want to avoid drastically increasing national contributions to Brussels.

Supporters of the gambling levy point to the fact that it would rake in over €13 billion throughout the next budget cycle and — for some, more importantly — address a serious public health issue. An estimated 80 million adults globally have experienced a gambling addiction, according to experts.

“We look on it [gambling] as an illness. It’s something that’s inborn in you and that can be ignited,” Shilton said.

Malta’s game plan

Malta has invested heavily in the gambling industry — including lotteries, betting and casinos increasingly operating online — which now accounts for around 12 percent of its gross domestic product.

These firms have relocated to Malta because of its light-touch licensing regime, business-friendly tax regime and balmy weather.

The country is “as dependent on the online gambling industry as Germany is on cars,” said an EU diplomat, granted anonymity to speak freely.

While gambling firms need local authorization to operate in most other European countries, securing the Maltese license is crucial to access banking services and gain a foothold in the EU market.

Malta-based firms dominated the German and Austrian online gambling markets before national regulators cracked down. This has prompted the Maltese government to refuse to recognize some court rulings and sanctions issued by other EU countries against its gambling firms.

Given its influence, it is hardly surprising that the gambling industry has found a friendly ear among Malta’s politicians in Brussels.

The Maltese president of the European Parliament, Roberta Metsola, last year gave the opening speech at an international gambling conference in Rome that also featured Italian Foreign Affairs Minister Antonio Tajani.

“I’m more than a little proud that it started in my island home of Malta,” she said, referring to SiGMA, a Maltese events company that focuses on online gambling founded by Eman Pulis, a university friend of Metsola.

Betting lobbies say they oppose higher gambling rates on the grounds that they will fuel appetite for the illegal market, away from the grasp of EU rules.

“A higher tax would lead to worse odds for the customers … and it is relevant because access to the illegal markets in Europe is, obviously, one click away,” said secretary general of the European Gaming and Betting Association, Maarten Haijer.

Nicola Matteucci, an economist at the Università Politecnica delle Marche in Italy who has undertaken extensive research on the gambling sector, argued there is a “point where prices exceed a certain level and the demand [for gambling] diminishes. But it’s not as immediate as suggested by the industry.”

Matteucci said that most gamblers will be undeterred by slightly higher taxes and worse odds as they are not fully rational consumers.

Anti-gambling groups reason instead that higher taxes will reduce the sector’s spending on commercials, preventing would-be punters from getting sucked in to gambling in the first place.

“Higher taxes will therefore mean less gambling advertising overall and many people would regard that as a public benefit,” said Derek Webb, the founder of the Campaign for Fairer Gambling advocacy group.

Club Med joins Malta

Malta has joined forces with fellow Mediterranean countries — Italy, Portugal and Spain — to challenge the mooted tax which was first proposed by the Parliament’s socialist lawmaker Victor Negrescu, said four diplomats with knowledge of the discussions.

According to the European Commission’s estimates, seen by POLITICO, a 3 percent tax on the net turnover of the online gambling sector would generate an estimated €1.9 billion per year.

With its big online gambling market, Spain is expected to be among the biggest financial losers, should the tax go ahead. It is estimated to be on the hook for €414 million per year, almost a quarter of the total amount. That compares to a projected bill of €165 million per year for Malta— a disproportionality high amount for such a small country.

Portugal is also reluctant to back the levy. It fears that higher taxes would eat into revenue brought in by state-run betting and lotteries that is currently channeled to the charity Santa Casa da Misericórdia de Lisboa‘s healthcare and youth support programs, said a Portuguese official.

Meanwhile, given the relatively low uptake of online gambling, Italy’s misgivings have surprised anti-betting advocates. Rome is expected to pay a mere 7 percent of the proposed new levy — a significantly lower proportion than its regular EU budget contributions.

However, Prime Minister Giorgia Meloni’s Brothers of Italy party has previously been receptive to the gambling industry. Last year its MPs passed a resolution encouraging the reversal of a ban on professional football clubs advertising gambling firms.  

LONDON — The U.K. capital’s transport authority has granted approval for Wayve and Uber to begin giving rides to members of the public in autonomous vehicles.

Transport for London (TfL) said it licensed 15 modified vehicles operated by the companies, which have a partnership, as “Private Hire Vehicles” (PHV) on a trial basis.

In a statement, London-based startup Wayve said the licenses were “an important step forward” that will allow it to begin giving rides to a small number of passengers later this summer ahead of a full public launch.

Wayve said its vehicles “are designed to operate autonomously, and will do the driving,” though under TfL’s rules, a licensed PHV driver must be present and responsible for the vehicle at all times.

“Safety is our top priority,“ a TfL spokesperson said. “Any new vehicle licensed to carry passengers on London’s roads must align with our Vision Zero goal of eliminating all deaths and serious injuries from collisions on London’s streets by 2041.”

Successive U.K. governments have sought to make the country a European pioneer in self-driving technology.

The Department for Transport opened a permitting scheme for companies to operate commercial robotaxi services without a human driver in May. Applications for that scheme — which is separate from TfL’s PHV regime — continue to be assessed by central government with input from local transport authorities including TfL.

Getting licenses isn’t the only obstacle facing robotaxi services. A survey by the London Assembly’s Transport Committee this month identified widespread opposition to autonomous passenger vehicles among the capital’s inhabitants, with just 29 percent of Londoners saying they support the roll out.

Europe wants to kick its U.S. tech habit, but it’s struggling to detangle itself from American data analytics firm Palantir. 

The problem is, the company’s tools are already deeply embedded in critical sectors across the bloc like policing, national defense and health systems. A report released today also shows that the tech firm has found ways to pay minimal taxes in Europe. On the show, hosts Zoya Sheftalovich and Ian Wishart discuss European alternatives and why detaching is easier said than done.

Next, our colleague Seb Starcevic has interviewed Mediterranean Commissioner Dubravka Šuica. The role that was originally thought to be low profile has instead thrust the Croatian official into the limelight as various crises hit the Middle East and the Mediterranean.

Plus: Three new foods are being added to the EU’s list of products with geographic indicators. These are the labels given to products that can only be made in a specific region, like Champagne or … yes, you guessed it, halloumi.

Questions? Comments? Get in touch! You can message us or send a voice message to our WhatsApp here or at +32 491 05 06 29.

Leading artificial intelligence models from Anthropic and OpenAI created fake online personas and tried to deceive human coders into abetting a cyberattack during a recent safety evaluation, the U.K.’s AI Safety and Security Institute disclosed Tuesday.

It marks the latest case in which a powerful AI system has attempted a digital attack on an unwitting third party without direct prompting during such an evaluation — heightening concerns the powerful technology is advancing too fast for responsible oversight.

The disclosure is likely to ignite fresh calls in Washington and Silicon Valley for more rigorous regulation of the AI industry, particularly over frontier models with advanced capabilities to detect and launch cyberattacks. It comes just days after similar testing mishaps involving some of the same models from OpenAI and Anthropic sparked urgent calls for new AI safety regulation and a push within Silicon Valley to slow the rapid pace of AI development.

Like its U.S. counterpart, AISI routinely conducts security evaluations to better understand what dangers both new and soon-to-be-released AI models pose to public health and safety. But even the digital security body said the actions it uncovered by Anthropic’s Claude Mythos 5 and ChatGPT 5.6 — the latest publicly released model from either AI lab — were unlike anything it had seen before.

“This is the first time AISI has seen deception of this severity that was targeted at a real person, unprompted, in the real world,” AISI said in a 35-page technical report accompanying a blog post Tuesday.

AISI said the malicious activity began on July 25. The AI safety evaluator detected it and launched an investigation on July 28, when it spotted “unusual data transfers” stemming from a cyber evaluation it was conducting on Mythos 5 and ChatGPT 5.6.

Overall, AISI determined that Mythos 5 and ChatGPT 5.6 took “autonomous, unsanctioned action on the live internet, targeting real people and organizations” during 10 of the 122 evaluations the research body conducted and isolated for review. The majority of those actions, it said, involved Mythos 5 and arose during a “single, sustained line of activity.”

In the most alarming incident it uncovered, Mythos 5 attempted a supply chain attack — a sophisticated technique often deployed by North Korean and Russian cyber operatives, in which hackers bury malware inside legitimate code.

To do so, Anthropic’s model created “multiple fake identities” on developer platform GitHub and used them to send messages “pressuring” an open-source software engineer to unwittingly introduce a bugged update into code widely available on the popular site, AISI said. When that effort failed, the AI “edited its earlier activity to appear harmless” and “considered adopting a fresh identity to continue,” AISI added, a sign the model was intent on repeating the ruse.

As part of the same effort, Mythos 5 also sent direct messages over GitHub to software engineers that contained malware.

In another sign of deceitful behavior AISI uncovered in its investigation, multiple AI agents it was testing appeared to communicate with one another about how to convince real engineers using GitHub to trust them. “One agent left public messages on GitHub offering collaboration with other agents working on the same challenge,” AISI wrote.

AISI’s blog and technical assessment make no mention of whether the models also attempted to exploit previously unknown software bugs — called zero-days — during the evaluation.

Last month, OpenAI disclosed that GPT 5.6 and another of its models escaped onto the open internet during what was supposed to be a controlled test, and then hacked another company in a first-of-its-kind, autonomous breach.

In response, Anthropic launched an investigation into whether any of its models took illicit action during recent testing and discovered Mythos 5 and two other models had hacked three organizations during tests dating back to April.

In a statement, an Anthropic spokesperson said they are “grateful” to AISI for their leadership and that this review underscores the need for “a broader conversation about how to safely evaluate increasingly capable AI agents.”

The spokesperson added: “As we shared after disclosing our own incident last week, the field needs stronger, shared standards for how evaluation environments are built and secured. We look forward to partnering with the UK AISI to learn more about this incident as we conduct our own investigation.”

An OpenAI spokesperson referred POLITICO to a blog post about the incident that went up Tuesday evening. “We are committed to working across the industry to strengthen shared practices for conducting high-risk evaluations safely, including convening stakeholders such as national AI institutes, independent evaluators, other AI labs, and other groups in the coming weeks,” the blog read.

AISI stressed in its blog that the malicious activity it disclosed Tuesday took place under “deliberately permissive conditions” so they could assess the safety risks posed by the two models. This included granting the models access to the internet, unlike the earlier incidents detailed by Anthropic and OpenAI.

AISI also noted the models were intentionally stripped of internal guardrails that block malicious behavior. AISI was only able to disable those controls because of its role testing Mythos 5 and ChatGPT 5.6.

Still, AISI said the incidents highlighted the need for greater monitoring of model behavior during testing, and tighter controls over their access to the internet.

The Trump administration is finalizing a voluntary framework under which AI labs would submit powerful models they want to release to the public for federal safety testing. But it has not yet made the framework public, and it includes no provisions for models AI labs are developing internally.

The incidents last month from OpenAI and Anthropic both involved models not intended for public release.

Some cyber experts say recent incidents highlight deeper questions around AI development, such as who is liable when AI systems break federal hacking laws.

“If any of these were human-originated, they would lead to clear and vigorous prosecution. I think it’s time for a serious discussion about updates to existing computer security law,” said Marc Rogers, a hacker and prominent cybersecurity expert.

Unpublished author Dylan Reed created a service that cryptographically seals manuscript drafts as evidence that a human wrote them. Vellumproof stamps each version with the time, then cryptographically links it to its predecessor. If you slip a draft in later or alter its date, it will break the chain of hashes. — Read the rest

The post The startup selling proof that a human wrote your manuscript appeared first on Boing Boing.

Nanit's flagship product is a camera bolted above a crib, logging the exact moment a baby's eyes open and close and scoring each night's sleep from zero to 100. The firm that built its brand identity described the job, per Sapna Maheshwari in The New York Times, as "transcending the negative connotations of 'surveillance.'" — Read the rest

The post Baby monitors now want to predict who your kid will become appeared first on Boing Boing.

Lokesh Dhakar's ASCII Today renders your text as big block letters made out of slashes, pipes, and underscores — the kind that head old README files and source-code comments. When you type a word at ascii.today, all 50 fonts appear at once in a grid, redrawing character by character.  — Read the rest

The post Type a word, get it back in 50 ASCII art fonts appeared first on Boing Boing.

Both Washington and Silicon Valley are in the midst of a collective freak-out over China’s recent advancements in artificial intelligence.

The latest round of consternation was triggered this month when a little-known Chinese AI startup called Moonshot released a new large language model called Kimi K3. The conventional wisdom had been that the leading AI models developed by companies like OpenAI and Anthropic were between six to 12 months ahead of their Chinese competitors. Kimi dashed those assumptions: now, analysts say American companies may be as little as two to three months behind. 

Dean Ball, a former Trump administration official now with OpenAI, warned in a bleak post on X that models like Kimi K3 could lead to a world of “full AI communism” and a “dystopian hellscape” of AI under full government control. 

Policymakers have worried for years now about China gaining an edge over the US in the AI race. Both the Donald Trump and Joe Biden administrations took steps to slow China’s AI progress, including blocking the export of the most advanced US semiconductors.  

The White House is already reportedly considering taking steps to ban “open-weight” models — models that are easier to adapt for a user’s own purposes — like Kimi K3 in the United States. The Trump administration has also accused Moonshot of using the unauthorized “distillation” of one of Anthropic’s models — basically using another model’s outputs to train itself rather than raw data — as well as gaining access to blacklisted Nvidia chips in Thailand.

But often lost in the debates about what to do about China’s accelerating AI capabilities is the question of why the US cares about this at all. Obviously, the American companies developing the latest frontier models care about maintaining their edge, but why should it matter to Americans if the chatbot in their pocket was developed in Silicon Valley or Shanghai? And perhaps even more so, why should it matter what chatbots people in Nairobi or Brussels are using? 

The concerns in the US about Chinese AI generally fall into three broad buckets: cybersecurity concerns; military and national security concerns; and human rights or democracy concerns.

For the moment, concerns about who is winning the AI race can feel a bit abstract, but as AI becomes more embedded into governments, militaries, and ordinary people’s lives, the difference will start to be felt in a much more material way at both a national and personal level. In general, there is a growing sense that it matters which of the world’s vastly different superpowers builds the technology that could transform everything. 

“People’s relationship with AI is becoming foundational to how they live their lives, so the choices people make about whose model they use and where they are physically hosted, as they share some of their most intimate secrets and ask for life advice and business guidance, and run an increasing share of their life — those are incredibly important,” said Ryan Fedasiuk, a former State Department technology adviser now at the American Enterprise Institute. “It’s a contest between the United States and China to define the operating systems through which people live and work.”

Here’s what else America loses if it loses that contest.

Chinese AI could be more vulnerable to cyberattacks 

The concerns about using Chinese AI are in some ways a repeat of the concerns over Huawei, the Chinese telecoms firm that built much of the world’s 5G internet infrastructure, but which the US government banned from operating in the United States during the first Trump administration over concerns that the Chinese government could intercept information transmitted over these networks. 

Today, the concern is that many firms are increasingly integrating Chinese AI models into their systems, both because they are often cheaper and because they are “open-weight.” (“Weights” refer to the setting an AI model uses to process a user’s inputs. “Open-weight” models make these publicly available for users to tinker with, rather than charging for access.) 

There are some indications that Americans using Chinese AI models are already vulnerable. A Booz Allen study from earlier this year tested four Chinese models commonly used by US developers and found that three of them generated software with far more “hidden vulnerabilities” that could be exploited by hackers than their US counterparts. There’s no proof that the models were doing this intentionally, but the study did find that the models were “changing their behavior depending on who the user seemed to be or what country the request referenced.”

AI can also be used to carry out cyberattacks. Although nearly all the leading models have safety protocols meant to prevent this, they’re not bulletproof. Even Anthropic’s Claude, generally considered one of the most secure models, was adapted by Chinese hackers last year to engage in cyber espionage. The open weights of the leading Chinese models could make it even easier to strip out the safety protocols. 

AI could give China a military edge

The simplest and most obvious argument for why AI matters for American national security is that it’s all too conceivable that the US and China could be at war in the years to come, and AI could be a major factor in determining who wins. 

The conflicts in Ukraine, Gaza, and Iran have shown that modern militaries are already extensively using AI for intelligence collection and targeting. Semi- or fully-autonomous drone swarms are a major component of US plans for repelling a Chinese invasion of Taiwan. Then there’s the risk of AI being used to generate new bioweapons or other dangerous threats. 

US experts believe China has pursued a “military-civil fusion” strategy, encouraging the People’s Liberation Army and Chinese defense contractors to collaborate closely with civilian technology companies and research institutions in order to gain an edge in military AI applications like intelligence analysis and drone swarms. It’s difficult to know exactly which of these capabilities China is focusing on, but procurement data suggests leading Chinese technology firms like Deepseek and Alibaba are involved in work with potential military applications. Analysts also accuse China of using outputs from US models like ChatGPT and Claude to train AI systems that could help develop China’s defense capabilities. 

And that’s just conventional weapons. The US government has alleged that Chinese labs have “continued to engage in biological activities with potential [bioweapon] applications” amid concerns that artificial intelligence could help make such weapons more sophisticated and deadly. 

China could export digital authoritarianism

Last year, it was reported that Miiloo, a fuzzy children’s plush toy with a built-in AI chatbot, would, if prompted, happily tell users Chinese Communist Party talking points like “Taiwan is an inalienable part of China.” The hubbub over Miiloo reached the US Senate floor. While it’s hard to imagine that many users were really asking Miiloo to help clear up East Asian territorial disputes, the affair illustrated much larger concerns about the dangers of letting AI models built by an authoritarian government with one of the world’s strictest censorship regimes become the global standard. 

Chinese generative AI tools are legally required to uphold the country’s “core socialist values,” according to a document published by its national cybersecurity standards committee. So it’s little surprise that DeepSeek, the Chinese chatbot that sent shockwaves through the US tech industry in 2025, politely declines to answer when you ask it what happened on June 4, 1989, in Tiananmen Square. 

It’s not just that Chinese AI could help shape the political narratives absorbed by billions around the world, at a time when US soft power is ebbing and surveys show people in many countries already now have a more positive view of China than the United States.

 The Chinese government is also increasingly integrating AI into its own censorship and surveillance apparatus, and is exporting tools like facial recognition technology to other authoritarian countries. 

The fact that under Xi Jinping, China’s government was centralizing power and becoming more, not less, authoritarian in the years leading up to the recent advances in AI are a major factor driving the mistrust in its technology. 

“I think many of the sincere arguments about the risks of these models and what China would do with them stems from the coercive authoritarian approach of China’s current leader,” said Mieke Eoyang, former US  deputy assistant secretary of defense for cyber policy. “I don’t think we would be having this conversation in the same way with someone like [China’s previous leaders] Jiang Zemin or Hu Jintao.”

It is a serious concern if models built to conform to the values and political priorities of China’s current government become the global standard. But some are skeptical of the idea that human rights and democracy should be the goal of AI competition, worrying that the damage has already been done. The premise of that idea has gotten “shakier in recent years,” says Steven Feldstein, a senior fellow at the Carnegie Endowment and author of the book The Rise of Digital Repression. Under this administration, the US has cut support for democracy and human rights programs overseas, and often allied itself with authoritarian governments. Then there’s the fact that at least one leading chatbot often seems to mimic the racist and antisemitic views of the world’s richest man who is also an ally of the current president. 

While it’s still true that Chinese AI reflects the authoritarian values and priorities of China’s leaders, Feldstein notes, “this idea that the US is standing at the forefront of protecting and advancing democracy, human rights, that we’re not sort of there to manipulate information or to push a narrative agenda that reflects the ideological preferences of its leaders, has started to fray.” 

The race to AGI 

There’s also a set of concerns around the topic of “artificial general intelligence,” the hypothetical point at which AI exceeds human capabilities and is able to improve itself. The concern, expressed by both US government commissions and senior officials in both administrations, is that China is “racing” toward AGI and that whichever country achieves it first will have a massive geopolitical advantage. This is the type of thinking behind invocations of the nuclear-era Manhattan Project to justify massive government investments in AI development. 

Chinese leaders do not appear to view AI competition this way. “The US conversation around this is much more ‘AGI-pilled’,” says Jeffrey Ding, a professor at George Washington University and expert on US-China technology competition. “The concern here is that we are very much on the brink of this explosion of more and more powerful AI that leads to it dominating everything.” Chinese leaders, on the other hand, “generally see AI as a productivity tool.”

If Chinese AI is a problem, what should we be doing about it? 

This is not just a Beltway or Silicon Valley concern. A recent Pew survey found that 43 percent Americans believe it is very important for the US to remain the leader in AI development, versus 22 percent who said it was not that important. Interestingly, the survey also found that most Americans believe China is already ahead on AI, though the expert consensus is that it’s still slightly behind. 

“We’ve gotten so used to the fact that the US has been the leading player in technological revolutions from like mobile internet to the internet era, so it’s worrying to feel we may no longer have that dominant strength,” said Selina Xu, China and AI policy lead in the office of former Google CEO Eric Schmidt. 

Even if there’s some consensus that AI competition is a priority, there’s less agreement on how to go about it. The challenge, Xu says, is “How do you manage the very concrete national security risks that come from competing with China on AI, but not turn technological competition into blanket protectionism?”

Often, the policy responses to this challenge have been contradictory. 

The Trump administration, in its first term, pioneered the policy of restricting the export of the most advanced semiconductor chips to China, but Trump undermined that policy last year by permitting Nvidia to sell its advanced H200 chips there. The move flummoxed China hawks in Washington and went against the preferences of AI developers like Anthropic, but probably had a lot to do with lobbying by chip maker Nvidia’s Jensen Huang, CEO of the world’s most valuable company. 

In some cases, the US may be inadvertently making China’s models more appealing. In June, the Trump administration placed export controls on Anthropic’s advanced Fable model. This move prompted the company to take the model down for all users and led to the first time that AI capabilities meant for the global public took a step backward.In response, French President Emmanuel Macron warned, “We will not buy any model made by [US AI] companies if from one day to the next you can just turn off the switch.” Chinese models are hardly immune from concerns about kill switches or back doors, but if both governments involved in the AI race are seen as meddling, customers may just opt for whichever one is cheaper. 

The latest flashpoint in the debate concerns the reports that the administration is considering banning open-weight models.  This prompted an open letter from dozens of leading tech companies including Nvidia and OpenAI defending access to these models as necessary for helping the US maintain AI leadership. Advocates note that open-weight models can help respond to vulnerabilities as well as create them: When a rogue OpenAI model recently hacked into the startup Hugging Face’s systems, Hugging Face’s engineers used an open-weight model developed by China’s Z.ai to analyze the attack. 

Despite the frequent comparisons, AI is not a national security competition like the early days of nuclear weapons or the space race. It’s a technology with potentially grave national security implications, that’s also used by millions of people around the world to plan their Tuesday night dinner or help with their homework. The log-in for Claude is not carried by a military officer at the president’s side. And much of the important work on developing these new technologies is being done by private tech companies, not government labs or defense contractors. 

It may be that AI capability will help determine which country has the edge in the 21st century. It may also be that the benefits of these capabilities will be shared: Chinese companies might be no less capable than their American counterparts when it comes to developing new medications or clean energy technology. 

The challenge of crafting technology to prevent a “dystopian hellscape” is to not accidentally make the existing world worse. 

Hi readers! Shayla Love here, science journalist and longtime fan of Your Mileage May Vary. I’m honored to be subbing for Sigal Samuel while she’s out on parental leave. I’m diving into your questions as a way to help understand human nature and our choices through multiple lenses: philosophical, psychological, and beyond. Please send in any emotional, body/brain, sociological, perceptual, or other kind of life quandaries you might have.

I’m swimming in information. I have tracking apps to keep tally of my daily steps, minutes online, my calories, my sleep. Throughout the day, I am awash with data — some “actionable” and some useless. I find it a struggle to prioritize. I find myself looking up the latest betting odds for a Senate race in a state where I don’t live. (I didn’t bet on the race.) What I want to know is: What should I know less about? I’ve heard that ignorance is bliss, though I don’t often find that to be the case. But maybe I know too much, or…too little about how much to know?

Dear Un-blissfully Aware,

As a fellow know-it-all, I relate to your impulse to gather as much information as possible. I used to obstinately reject the idea that ignorance was bliss. Even if I learned something that was unpleasant, wouldn’t it be much worse not to know it? 

But, as you may suspect from sharing my temperament in this arena, this approach to life can lead to hoarding knowledge like a frantic animal preparing for winter. You’re acquiring information as if it’s a scarce resource (which it’s not) and as if choosing to know something is neutral (it isn’t). What helped me understand the implications of this sort of approach was learning about the cases when people decided not to know, or the study of “deliberate ignorance.” 

Several years ago, a psychologist, Ralph Hertwig, and lawyer, Christoph Engel, who both work at the Max Planck Institute in Berlin, examined what happened in the early 1990s, when the archives of East Germany’s secret police, the Stasi, opened to the public. Any person who had been living in the German Democratic Republic could check if they had been spied on. There was a pretty substantial catch, though: The spies were often “unofficial collaborators” — friends, family, teachers, or lovers who had been tasked with covertly collecting information. When the files were opened, many elected not to look. Hertwig and Engel estimated that potentially more people chose not to know what their files contained than those who did. 

This goes against what we think we know about humans and how you have described yourself. Even Aristotle seemed quite certain that “all men naturally desire knowledge.” In the 1980s, the psychologist George Miller followed along in this line of thinking. He described humans as informavores: creatures with minds that constantly seek out and consume information. So, it can feel surprising to hear about occasions when people didn’t start grabbing for every available datapoint like squirrels dashing for acorns as the first winds of fall arrive. 

But, as Hertwig and Engel — who collaborated on an entire book called Deliberate Ignorance: Choosing Not to Know — point out, we’re surrounded by people choosing ignorance all the time. They avoid looking at their bank statements, they skip doctor’s appointments, and they cover their ears and say “no spoilers” if someone is talking about a movie they haven’t yet seen. 

Even Nobel Laureates do this. When James Watson, one of the co-discoverers of DNA’s double helix, had his own genome sequenced, he requested that a specific gene be left out: ApoE4, which can reveal an increased Alzheimer’s risk. Watson’s grandmother had Alzheimer’s, and her experience clearly made an impression on him. (Then, many in the scientific world tried to remain ignorant to Watson’s increasingly misogynistic and racist comments, until it was unignorable.) 

Why do people choose not to know? The most common reason: Ignorance can be an extremely effective way to regulate any emotions that might crop up in response to knowledge. You can minimize negative feelings by, for example, not knowing that your otherwise friendly neighbor had been reporting on your comings and goings. 

In a study about romantic relationships, a majority of people said they didn’t want to know if their partner had ever thought about cheating on them (but never acted on it). Other research showed that most people wouldn’t want to know the exact time of their death. Alternatively, someone could cultivate the feelings of joy and surprise by waiting to find out the sex of a baby until it’s born.

Deliberate ignorance can also be used to protect us from our biases. Scientists do blinded studies, because the knowledge of what drugs are being used can unwittingly change the outcomes or research. When orchestras implemented blind auditions, more women musicians were accepted. 

Of course, deliberate ignorance can be problematic when it causes harm to yourself or others, like failing to gather basic information about your financial or physical health. In some cases, for instance, about 10 percent of people who got tested for HIV didn’t come back for their results — which could lead to further spread of the disease. 

But when skipping out on knowledge doesn’t endanger you or those around you, it’s worth asking: How would this information make me feel? 

This sounds like a question that’s missing from your information-ingestion habits. You don’t have to — and shouldn’t — ignore everything that comes across your desk. But your deliberate ignorance filter is turned entirely off; seemingly anything passes through it. To take just one of your examples, you mentioned fitness wearables. It can be useful to be aware of your general activity levels and your sleep, but honestly, don’t you already know enough from your memories of a workout class and how rested you feel in the morning? A quest for more detailed knowledge like this can backfire, making people more anxious, and disconnected from their actual physical experience. 

It’s not just about how much knowledge to take in, it’s also about what kind. The breadth of information that you’re seeking out makes me wonder if you’re not already using knowledge to regulate your emotions by favoring superficial knowledge over the meaningful. When we face difficult tasks (even if they’re important, and we want to get them done), that’s usually when, suddenly, a Senate race in a far-off state starts to become interesting, or you find yourself scrolling a high school acquaintance’s wife’s Instagram. 

In the 1990s, two technology researchers proposed the concept of “information foraging,” which was inspired by early user behavior on the web. A few years later, Webster’s dictionary announced that their word of the year was “infosnacking,” or mindless grazing for useless knowledge online in order to pass the time. 

Snacking sometimes is fine, but you’ll start to suffer if you only eat chips for every meal rather than something more nutritious. I suspect you might be avoiding the slightly more challenging task of taking in other, more nutrient-dense knowledge. If you cut out the step tallies, screen minute totals, and news headlines scrolling, what could you choose to know instead? 

What I’ve learned about myself is that I will always lean towards being an informavore. I can’t help it. I am hungry to know, and there is no need to totally overhaul this valuable part of who you are (as I also tell myself!). This means you don’t need to replace your infosnacking with zenfully staring at the ceiling or emptying your mind through meditation. You can still put new things into it! But you could worry less about your daily information calorie intake and spend more time thinking about the nutrition content of your knowledge diet. 

Take a lesson from our many deliberately ignorant friends, and try disregarding some of the little stuff — the body tracking, the random factoids — for a period of time. But, at the same time, increase your knowledge about other subjects: a neglected hobby or a nonfiction book collecting dust on your shelf. Have a long conversation with your best friend and ask each other questions you’ve never asked before or interview an older relative about their childhood. 

And remember that all of this knowledge relates back to your emotional life. Rather than asking yourself what you should or shouldn’t know, examine how you might be using knowledge to alter how you feel. Knowledge changes us. It can make us happy, anxious, excited, or sad; it impacts our decisions and how we see ourselves and other people. 

In some of your newfound spaces of intentional ignorance, you may find just some surprising moments of bliss.

Bonus: What I’m reading

• The ostrich may be famous for its head-burying ignorance, but that’s actually a myth dating back to the ancient Romans. Ostriches don’t hide their heads, but, rather, bury their eggs underground. I highly suggest flipping through the book Ostrich by Edgar Williams, professor of cardiopulmonary science at the University of South Wales, for a natural and cultural history of our largest living bird that is way more interesting than it needs to be. 
  
• In a moving essay in the China Books Review, poet and writer Zhang Er remembers growing up during the Cultural Revolution and receiving “torn books,” or sections of forbidden books, to read from a family member. Even with incomplete knowledge, “my world expanded with each torn book,” she writes. 
  
• Not a book, but a powerful story on the ripple effects of being exposed to new information in East Germany: the 2006 movie The Lives of Others, from director Florian Henckel von Donnersmarck, which I recently saw for the first time. A Stasi officer listens into the life of a playwright, whose own fragile ignorance about his situation in the GDR is becoming challenged.

The internet may no longer be solely the domain of humans. Last week, OpenAI disclosed an “unprecedented cyberincident”: An experimental AI agent successfully hacked its way into the open internet.

Specifically, the agent was assigned a task; in order to complete it, the agent broke out of an isolated research environment and hacked into a third-party platform called Hugging Face. It’s a move that many experts deemed inevitable, given the speed and scale of advances in AI technology — and it raises serious questions about AI safety.

But for Konstantinos Komaitis, a senior fellow with the Democracy and Tech Initiative at the Atlantic Council, it wasn’t the unexpected behavior of the AI that was significant. It was what that behavior could mean for the internet’s fundamental, decentralized infrastructure and whether it would spur calls to build new barriers against autonomous AI agents. 

Komaitis argues that such barriers are not the solution, however. He spoke with Today, Explained co-host Sean Rameswaram about why an open internet is actually key to combating AI cybersecurity threats.

Below is an excerpt of the conversation, edited for length and clarity. There’s much more in the full podcast, so listen to Today, Explained wherever you get podcasts, including Apple Podcasts, Pandora, and Spotify.

So most people see that this happens and they think, “Oh no, AI went rogue. How long before it kills me?” You see that this happens and you start thinking about infrastructure. Tell us more about why you were thinking about infrastructure in light of this AI agent breaking containment.

The internet was never designed with full security in mind, right? 

When you’re creating a decentralized system, you cannot possibly foresee every security or vulnerability that might come up. But because you have a system that is based on building blocks, you have the extraordinary capability of actually addressing security issues as they come up through those building blocks without breaking the whole system down. 

And of course, the other thing that this does is that it pushes you towards collaboration, because when you have so many building blocks, you cannot possibly possess all the knowledge for each building block. So you’re bringing literally everyone to try to address these problems. 

Take the internet, for instance: We have spent decades addressing those vulnerabilities and developing mechanisms to authenticate users and devices, encrypt communications, mitigate distributed attacks, coordinate incident response, and of course share threat intelligence. 

Now, what is new with agentic AI is not that simply the malware is better or the phishing attacks are more sophisticated. It’s the emergence of systems that can actually discover vulnerabilities across thousands of systems. They can reason about alternative paths to an objective. They can adapt when they’re blocked. They can chain together legitimate internet services in unexpected ways. Then they do that while they’re operating continuously at machine speed. And this is really at a scale that the internet is not ready to necessarily cope with. 

Effectively, the internet’s openness becomes both a strength and a vulnerability. So the internet was optimized for interoperability, and AI now is optimized for exploiting that interoperability.

And what scares you the most about that? What do you think is most vulnerable to threats?

The fact that we do not have the appropriate mechanisms and institutions to be able to deal with that. I come from the internet world. I’ve spent 20 years of my career defending the open internet and discussing it in international fora. And one of the things that a lot of people underestimate about the internet is how valuable trust is as a property within the system. 

We are talking about networks that exchange data literally based on trust. So what really concerns me right now is that in many ways, we are asking 21st-century AI systems to operate on 20th-century assumptions about trust. And unless we figure that out and we realize it, we will continue having these problems. And of course, the knee-jerk reactions that are coming with this, which are, “Let’s fragment the internet, let’s restrict it, let’s restrict access, let’s take control over it.” That is never the solution.

What do you see as the solution?

Effectively, we need to build institutions that are trusted and are able to cope with those incidents as they happen. Because right now you have OpenAI and you have Hugging Face telling everyone, “Don’t worry, we’ve got this.” And we don’t know; they might have this. But at the same time, I cannot help but wonder. And many, many other people have wondered whether, actually, this is very good PR for these companies and especially for OpenAI.

OpenAI just went to the world saying, “We have developed one of the most powerful LLMs, and we realized that it behaved the way it behaved, but don’t worry, we are going to fix this.” And in this current climate and in this current timing, I am not sure that this is enough. You need institutions that are much more transparent, much more accountable, and much more collaborative across the board.

You want institutions to step up and essentially serve as a watchdog. Help us understand which institutions, because in the United States, famously, our government has done very little to regulate tech.

First of all, we need to stop thinking of institutions as necessarily government-affiliated, right? Or that they are the outcomes of government initiatives. There can be in collaboration with governments, but one of the things that the internet has taught us is that institutions that are built through a bottom-up coordinated process have the tendency of actually being more agile and able to deliver some of those things that we’re talking about. 

So take, for instance, again, open standards. The internet’s open standards are not created by any agency, government or private. It’s created by institutions where engineers from all across the board and all over the world gather together and create those standards.

That’s reminding me of the original design of OpenAI to be this not-for-profit company that had everyone’s best intentions in mind, that could do something idealistic and moral and ethical because all of the profit-minded companies weren’t going to. And now look at OpenAI. Their not-for-profit arm is an afterthought, and they’re chasing profits. 

Do you think it’s practical to leave this to institutions? Because what we’ve seen so far is that institutions bend toward capitalism.

It really depends on how you build the institution, right? It really depends on what sort of guardrails and checks and balances you have around it. In order to build an institution, you need to really know what you want to achieve. You need to have a north star. 

One of the reasons the internet worked was because everybody disagreed, but they agreed on the common shared goal, which was to connect people across the world. For AI, we still do not have that northern star. And once we get it, that’s when you start the building of those institutions in order to facilitate this and bring everyone together.

For me, it is very important for everyone to understand that keeping an open internet is really more important than ever, especially as AI agents become increasingly capable. Because it is tempting to think that the answer to new AI risk is literally ‘build more barriers.’ But the internet’s greatest strength has always been its openness. So the challenge today is not that the internet is too open; it’s that its trust architecture was designed for a world in which humans or software directly controlled by humans were the primary actors. 

Now, it’s being challenged by this agentic AI that introduces a new type of participant — systems that can reason and plan and act with limited human oversight. So we need to evolve our understanding of trust and what it means online. And that will require a lot of work because, as you know very well, Sean, it’s very difficult to build trust, but you can break it within seconds.

AI is the hottest thing in medical care right now — but many of us feel trepidation about it. Just one illustrative public survey sample: An October 2025 KFF poll found just 8 percent of Americans reported feeling a “great deal” of trust in AI managing their appointments or analyzing their health records, and only 32 percent said they would trust an online health tool that uses AI to access their medical records to provide personalized health information.

But many clinicians and healthcare administrators see AI as a powerful new tool that offers myriad opportunities to streamline and improve treatment. A 2026 survey found that more than 80 percent of US doctors use AI professionally — doubling the share from 2023. Physicians are excited by AI’s potential to keep more accurate notes of interactions with patients, to act as a second pair of eyes for human doctors, and to monitor people at risk of deteriorating and ending up in a dangerous situation.

The disconnect between what people and their providers want from AI could create more distrust, at a time when faith in the healthcare system and the medical profession have slid. Patients today want to feel empowered and in control. How can that be possible when these seemingly godlike machines are becoming more and more entrenched in our hospitals and doctors offices?

The answer comes in four words: “human in the loop.” It’s the principle upon which the ethical integration of AI depends and it could help to bridge the gap between lay people and the professionals on AI in medicine. In surveys, people are much more comfortable with the idea of their doctor using AI as an assistant than with AI acting on its own. And most clinicians want to use AI in that way, as a second opinion or passive monitor, not as a replacement for their judgment. There are real fears among the healthcare workforce about that possibility: A group of NYC nurses who were recently laid off claim it’s because their labor was going to be replaced by AI. “Human in the loop” appears to be a point of agreement between doctors and patients at this pivotal moment.

“Doctors…and nurses and staff always have been interested in primarily making the best decision for the people under their care — and these tools can help with that,” Alison Callahan, a research scientist at Stanford University who works on AI programs used in the university’s health system, told me. “The interest in making sure those tools are accurate is high.”

But what does “human in the loop” really mean in practice? How can you know when and how your doctor is using AI? And what is the best way to talk to your provider about the sudden influx of artificial intelligence in healthcare before a robot starts taking appointment notes or analyzing your MRI? I called some leading experts to find out. 

How AI is currently being used in medicine

Patients and providers alike are incorporating AI into healthcare. Individuals are using commercial AI chatbots to ask about their symptoms or the health metrics tracked by their Apple Watch, while large academic medical centers are developing sophisticated programs and protocols to try to improve medical care at the population level.

It starts with ChatGPT, Claude, etc. — the large language models that are available to the public. People are increasingly turning to them to try to understand what’s going on with their own bodies. Individual physicians are also consulting with large language models to answer questions or get up-to-date on the latest research as they figure out how to best care for their patients. 

Then there are ways in which hospitals and doctors offices are adopting AI at the institutional level. Many facilities are using AI as a way to take, collate, and summarize notes on a patient; in theory, it’s a more organized way to keep track of the informal interactions and observations that doctors have when checking on their own patients. Hospitals are also using AI to handle some administrative tasks, like scheduling follow-up appointments; some health systems have even started to use AI to help patients get ready for appointments — to send reminders about colonoscopy prep, for example.

And finally, you have maybe the most ambitious use of AI by health systems right now: as a diagnostic and risk prediction tool. In these cases, AI might offer a second opinion when, for example, a doctor is triaging a patient in the emergency room. It might help the ER staff figure out how to prioritize patients. Or these programs could monitor people either during a hospital stay or out in the real world (by drawing data from the person’s wearable) and make predictions about who may be at higher risk of complications and require further care. AI could recommend that somebody would benefit from seeing certain specialists or receiving a specific medicine or lab test, and generally offer proactive advice about the patient’s medical care.

But at this point, AI adoption is still “highly localized,” said Jennifer Goldsack, CEO of the Digital Medicine Society, a nonprofit that works with healthcare providers, drug makers, and government agencies on how to incorporate new tech (including AI) into clinical care. It depends on the individual doctor or health system. A lot of them are setting up their own programs and their own protocols for how to use these tools.

That is a big reason why it is so important for patients to be proactive about understanding how AI is being used for their health care. You can’t make assumptions; the only way you’re going to know for sure is to ask.

The questions you should ask your doctor about AI

By and large, experts say, patients should feel confident: Doctors and nurses want to keep a human in the loop, even as they integrate AI into their workflows.

“It will be a doctor who is going to be reading that summary or a nurse who is going to be reading that summary and then taking an action to order a lab or put a recommendation in for a follow-up appointment,” Callahan said. “There is high interest in making sure that that is the right decision for that person. That hasn’t changed.” 

Still, many patients say they’d be more comfortable with AI use if their doctor fully explained it in advance. And health systems may have their own priorities that push their facilities toward more rapid AI adoption and delegating more tasks to these AI tools, as seen in the recent NYC nurse layoffs.

So if you want to be informed on exactly where this technology is present and have the ability to consent to its use, you have every right to ask your doctor, experts say. 

“AI is new, but the trust that serves as the foundation of the physician-patient relationship is not,” Timothy Keyes, a machine learning scientist at Stanford Health Care, told me over email. “To that end, I think that conversations about medical AI use should be open, honest, and transparent — just like any other conversations about shared decision-making in the clinical environment should be.”

For some things, your doctor should be asking you proactively if you consent to AI use — note-taking, for example. At my most recent primary care appointment, my doctor asked me if it’d be okay for him to use AI to take and summarize notes from our conversation; Goldstack told me she’d experienced the same at recent physician visits. (This is probably the most common AI use that you will encounter, and Keyes said it’s worth considering giving your consent: “There is growing evidence that they reduce physician burnout and save them at least a bit of time each day writing notes.”)

There are also a number of direct questions that you can ask:

• Will AI be used in my care and how?
• How is my data being protected?
• Can I opt out of any AI services that I do not feel comfortable with? (Keyes noted that patients should be allowed to opt out of any care, AI-related or not; if opting out is not an option, ask how a human provider will be involved.)
• How is the health system or clinic making sure that any AI system they use is working as intended?

And the transparency goes both ways. If you’re asking a question because you consulted ChatGPT before your appointment, tell your doctor. If you’ve talked with a chatbot because of mental health struggles, tell your doctor. And at the same time, feel free to ask your physician how you yourself could actually use AI in a responsible and productive way to improve your health.

“This opens up the opportunity for both the physician and the patient to be humans-in-the-loop,” Keyes said, “in different parts of the loop, with different perspectives, using an AI system to better understand the bigger picture.”

In a way, the novelty of AI and its rapid adoption is an opportunity for all of us to be nosier and more inquisitive patients. What all of these questions really come down to, Callahan said, is how your doctor is making decisions about your health care. That is relevant to all of us, no matter how AI is involved or even if there is no AI being used at all. 

Callahan said she always has a list of questions for her doctor when they recommend a course of treatment: “What are the factors in my health that are informing this recommendation that you have? Would you be making this recommendation for other patients who are similar to me? What can you tell me about the outcomes that I might expect to experience if I say yes to this?”

“I actually think if they can point to the part of your health that is connected to the decision, whether or not an AI tool helped to make that connection is secondary to their ability to communicate effectively to me about it, and help me to feel engaged in making a decision about my own care,” she said.

AI is changing medicine quickly, for both patients and their doctors. The best way to stay ahead is to talk about it.

When I first heard about the new Apple Upgrade program, which lets you lease devices like iPhones and MacBooks for a monthly fee, I was offended. It amounts to paying a tithe to one of the world’s richest companies just to borrow devices for a couple years, rather than buying them outright. You could then choose to purchase the device, which is outdated at that point, or upgrade and keep paying that monthly fee. You may never own an iPhone again.

Then, as my mind wandered to the stack of old phones in my closet, it occurred to me: What’s so great about owning these things to begin with? 

Apple, of course, would love to sell you a new iPhone for keeps. Its most advanced model, the iPhone 17 Pro Max, will set you back $1,200, a price that’s expected to rise soon due to the global shortage of storage and memory chips. You can sign up for an installment plan — most carriers offer these, as does Apple through its credit card — and pay it off in two to three years. Or you could lease the thing for $35 a month under the new Apple Upgrade program. You can pick a 12-, 24-, or 36-month lease, depending on the device, and you don’t get to keep the phone at the end of the term unless you decide to buy it by paying off the remainder of the retail price in one lump sum. (This is similar to the controversial rent-to-own model you find at places like Rent-a-Center.) 

For the financial side of the new program, Apple has partnered with none other than Klarna, the “buy now, pay later” giant. When you go to lease a new device, Klarna runs a soft credit check and decides if you’ll be able to cover the monthly payments. When I asked Klarna, the company did not tell me where it draws the line here, but it’s worth noting that critics have accused Klarna of a lack of underwriting and of lending to people with subprime credit scores. If you miss three consecutive payments, Klarna will terminate the lease agreement and possibly send a collection agency after you.

“How do I know people aren’t getting a good deal here? If they were, Apple wouldn’t be offering it.”

Aaron Perzanowski, University of Michigan law professor

While there was some speculation last week that Apple might lock people out of leased devices if they failed to pay their bill, Apple confirmed to me that it will not put limitations on device functionality due to missed payments or default. If you want to cancel the lease, you face an early termination fee. If you choose to keep paying the monthly fee, you can keep upgrading with new lease agreements for new devices every few years, existing in this cycle indefinitely.

“I don’t think people are getting a good deal here,” said Aaron Perzanowski, a law professor at the University of Michigan and author of The End of Ownership: Personal Property in the Digital Economy. “How do I know people aren’t getting a good deal here? If they were, Apple wouldn’t be offering it.” 

Buy an iPhone? In this economy? 

If you’re someone who likes to get a new iPhone or MacBook on a regular basis, Apple’s new leasing option might make a lot of sense. The monthly fee to lease these devices is cheaper than the payment plan to buy them, and electronics are depreciating assets. If you own one, you can sell it or trade it in for credit toward a new device, but they’re all worth less and less as time goes on. Furthermore, Apple eventually stops supporting old devices through software updates, so they might just stop working at a certain point. Put another way: You may own the phone, but you’re still just licensing the software that makes it work.

Renting an iPhone does sound bleak, though. The United States is suffering through an affordability crisis as prices across the board rise in the face of new tariffs and new wars. Meanwhile, AI is promising to transform the way we work if it doesn’t simply steal our jobs first, adding further insecurity, and the data center boom is making electronics more expensive. This era of economic anxiety is pushing people to use “buy now, pay later” services like Klarna and Affirm to pay for groceries or a tank of gas. (These companies faced scrutiny by state attorneys general a few years ago for operating like predatory lenders.) And now Apple, surely suspecting that many people can’t afford to pay full price for new phones, is inviting us to rent our devices at a monthly fee that undercuts the path to ownership. 

Apple could have just called this the Apple Rental program, by the way. Lease sounds nicer, though, like something you do with a car. 

“It is funny that they frame it as not a loan but as a lease,” Louis Hyman, a history professor at Johns Hopkins University and author of Debtor Nation: The History of America in Red Ink. He added that “leasing” has class implications, suggesting that you’re either someone who needs to have the newest things but can’t afford them, or that you’re so wealthy, you’re indifferent to money.

Suffice it to say, the bulk of people who will soon be leasing their iPhones are probably not the ones who are indifferent to money.

Apple adopts its final form

The new Apple Upgrade program is the company’s latest customer acquisition strategy. As the rising price of hardware has made cheaper Android devices or the refurbished market more attractive, Apple is offering upgrade enthusiasts and budget-minded users, including people who simply couldn’t afford to buy Apple products in the past, a deal to join the company’s ecosystem. After all, keeping people supplied with new iPhones and MacBooks also helps keep them subscribed to Apple services, like iCloud, which now makes the company more money than Mac, iPad, Apple Watch, and other accessories combined.

If Apple’s financial future hinges on getting more and more people to subscribe to these services, it’s only natural that the company would want to lower the barrier to entry. So Apple is betting that by letting people use but not own its products, it will extract more profit in the long run through lease payments and subscription fees. After all, it wasn’t that long ago that it seemed like nobody was interested in upgrading their iPhone, since the new phones looked so much like the old ones. Now, Apple is just trying to get everyone on autopay, effectively subscribing so that they get the latest devices when they come out.

There’s not necessarily any harm in giving people a cheaper way to access expensive but useful products. For more than a century, installment plans have enabled people to buy modern conveniences like sewing machines, radios, and eventually, televisions. Leasing is a popular way to keep yourself in a new car, sometimes with free maintenance. Meanwhile, cellular carriers have a long history of helping their customers buy phones. Nearly two decades ago, you could get an iPhone 3G for $199, thanks to subsidies from AT&T, which the company recouped in service fees over the course of your contract. Sprint and T-Mobile have even offered unlimited upgrades through leasing programs of their own in years past.

Apple previously worked with Citizen One Bank to offer loans to customers who wanted the option to upgrade their iPhones every year. The payments were higher and they included a fee for AppleCare, but every year, you could trade in your current phone for a new one. If you didn’t want to upgrade, you could simply keep paying the installments, and you’d eventually own the phone. Most carriers now give you the option to set up a payment plan to purchase a new device that simply amounts to the retail price of the gadget divided by the number of months you’ll need to pay it off, usually 24 or 36, with zero interest. That makes it easier to get your hands on an iPhone Pro Max, and if you pay it off in full, it’s yours for life — or until Apple convinces you to buy another new iPhone.

The difference between paying those monthly installments and paying a monthly lease agreement, of course, is that the former puts you on the path to ownership. The latter simply puts you on a path to make a decision: Do you want to buy the thing and recoup some of the money you’ve already spent, or do you want to keep making payments?  

“What ownership ideally gets us is independence,” Perzanowski said. “It gives us autonomy. It gives us the ability to function in the world without relying on third parties.” He went on to explain how moving from owning a product to leasing it means you’re stuck with that third party. “I’m tied to that manufacturer in a way where they get to exert a fair amount of control over my behavior,” Perzanowski said. “Historically, we’ve been primed, especially in the United States, to resist and reject that kind of control.”

One great thing about owning an iPhone or a MacBook outright is that if you lose your job to AI, you don’t have to come up with a monthly payment in order to keep using those devices to apply for new jobs. Another great thing about ownership is that should you need a couple hundred bucks, you can sell that old phone or laptop and pocket the cash. Maybe the best thing about owning these devices is that you can repair them and keep using them for many years — or at least until Apple stops supporting them. 

That doesn’t mean leasing never makes sense. If your digital life revolves around always having the newest devices and you upgrade every year or two no matter what, you might actually save money by doing so through Apple’s leasing program. If you need an iPhone or MacBook right away but can’t afford to pay full price or even cover the monthly payments on an installment plan, a one-year lease could be a good solution. 

Invariably, when you lease anything, you’re entering into a contract, one that comes with consequences if you break it. Leasing an iPhone means you’re tied not only to Apple but also to Klarna for the next 12 to 36 months. If something goes wrong — you lose your job, you lose or break your phone, or you simply don’t want the device any more — you’re subject to the terms and conditions of these big tech companies. If you keep renewing your lease, you may very well end up spending more on a phone than you would have if you’d bought it outright. That would be fine with Apple, of course. It has shareholders to please.

Correction, July 30, 1 pm: This story originally misstated how the previous Apple upgrade loan program worked; it allowed phone trade-ins every year, not every two years. 

The AI industry’s investors and critics don’t agree on much. But many in each camp share at least one basic conviction: America’s top labs are about to make a killing. 

Capital markets have signaled their faith in Anthropic and OpenAI’s impending hyper-profitability, valuing each at nearly $1 trillion. Many of Silicon Valley’s progressive adversaries also expect the labs to grow filthy rich but fear the implications, warning that AI-induced automation could transfer vast sums of money from ordinary workers to a handful of giant tech companies. Sen. Bernie Sanders’s call for nationalizing the top AI labs rests partly on that concern. 

But recent advances in Chinese AI call all of this into question.

Over the past two months, Chinese companies have released three AI models that are nearly as powerful as America’s frontier systems — and radically less expensive. 

In June, Beijing’s Z.ai debuted a model that performed nearly as well as Claude and ChatGPT’s second-tier systems on independent benchmarks. Weeks later, another Chinese firm, Moonshot, unveiled “Kimi K3,” a model that allegedly outperforms all of its American rivals except for the very latest versions of Claude and ChatGPT. Finally, just days ago, Alibaba launched a preview of Qwen3.8 Max, which purportedly outclasses even OpenAI’s most advanced systems, while trailing only Claude’s Fable in its capabilities. (Disclosure: Vox Media is one of several publishers that have signed partnership agreements with OpenAI. Our reporting remains editorially independent.)

These developments don’t merely threaten America’s AI giants with stiffer competition in the race for superintelligence. Rather, they raise a more harrowing prospect: that the AI race’s ultimate rewards will be far smaller than anticipated. In a world where new advances can regularly be leapfrogged by cheaper upstarts, hoarding the technology — and its profits — will be harder for any one company to do.

In other words, building a machine God might not be as lucrative as it’s cracked up to be. AI, it turns out, may “want to be free.”

How AI was supposed to pay off

To see how China’s new models threaten Anthropic’s profit expectations, we must first examine why those expectations have been so high.

This is not entirely self-evident. After all, AI labs aren’t much like the hyper-profitable tech giants of the 2010s. Facebook and Airbrb were relatively capital-light businesses with ultra-low marginal costs (adding a profile to Facebook or listing to Airbnb costs the companies virtually nothing). And once each gained a foothold in their respective markets, network effects enabled them to retain formidable positions without needing to constantly upgrade their products.

Building a state-of-the-art AI company is a much more involved — and astronomically more expensive — endeavor. To get to the frontier, Anthropic and OpenAI have sunk (at least) tens of billions into semiconductors, data centers, power plants, and other capital investments. Staying at the cutting-edge, meanwhile, compels them to perpetually churn out evermore costly models.

To put a new Claude model through its initial training — in which it spends months digesting the internet and sussing out statistical patterns within its text — can now cost hundreds of millions of dollars. And such foundational computation is only the beginning. A truly superlative model requires several additional months of fine-tuning. Armies of contracted experts — such as computer scientists, physicians, and mathematicians — tutor the models, grading their answers and guiding them towards better ones. Then the AI systems complete millions of rounds of practice, in which they learn through trial and error how to solve countless problems. This arduous process, known as “post-training,” compounds the costs of a single model’s development. 

All of which raises the question: Why would investors expect businesses with a cost-structure this challenging to be not merely profitable, but massively so?

There are (at least) two answers. The first (and most obvious) is that the market for superintelligent machines is liable to be vast. Frontier AI systems promise to reduce costs and improve performance in myriad white-collar sectors. And Anthropic’s soaring revenues indicate that firms do, in fact, find Claude useful. A company like AirBnB has earned billions by revolutionizing a single industry; imagine then what a technology that remade virtually all industries might be worth.

Of course, plenty of technologies are valuable but not massively profitable to produce. After all, in well-functioning markets, competition should eventually erode individual firms’ margins, even if the underlying technology continues generating huge value. 

But this is where the second answer comes in: Frontier labs’ immense costs are a burden, but they’re also a safeguard against competition — or, in industry parlance, a “moat.”

Startups may be able to afford to build or acquire more rudimentary models, many of which are “open source.” But, the thinking goes, they won’t be able to deliver Claude Fable-level performance without raising giant amounts of capital. And what investors will be willing to pour hundreds of billions into an AI pipsqueak that’s light-years behind Google, Anthropic, and OpenAI?

Alas, the Chinese AI labs’ rapid progress — and the way it was achieved — suggest that Anthropic’s moat may be shallower than previously thought.

How Moonshot swam Anthropic’s moat

The existence of powerful, Chinese AI systems is neither new nor surprising. Xi Jinping’s government has made vying for global AI dominance a key economic goal. And China’s DeepSeek, which also has stunned US companies with its lower-cost competitive models, surpassed ChatGPT as the most-downloaded free iPhone app more than a year ago.

The latest models, however, have dramatically narrowed the gap in capabilities between frontier American systems and their Chinese rivals. Just as critically, they’ve done so in a manner that other, relatively underfunded AI upstarts might be able to emulate.

Alibaba and Moonshot needed to invest massive resources to train their base models. But they allegedly found a low-cost way to refine those models into near-frontier systems: Just ask Claude.

Or, more specifically: Engage Claude in 16 million conversations, using 24,000 fake accounts. In each of those exchanges, ask the model to not only answer countless difficult questions but also, walk you through its reasoning, step by step. Then take all of this data and feed it into your own model as study material, training it to respond to the world’s most challenging queries as Claude would. 

Through this process — known as “distillation” — an AI lab can replicate virtually all of a frontier model’s capacities, without sinking vast sums into human experts and post-training computing runs. 

China’s AI labs have not admitted to using distillation. But OpenAI and Anthropic both reportedly uncovered Chinese distillation attempts earlier this year. And some of the new models appear to display tell-tale signs of distillation in conversations with ordinary users; Kimi K3 has routinely identified itself as “Claude.”

Chinese AI companies are hardly alone in using distillation to catch up with frontier labs. Earlier this year, Elon Musk admitted in court that xAI enhanced Grok’s capabilities by running distillation techniques on Claude and ChatGPT. Nonetheless, China’s latest models appear to demonstrate that distillation can help take a second-tier model to the frontier’s threshold.

America’s frontier labs have tried to defend themselves against such imitators. But this is technically difficult when distillers can assemble massive networks of bots, each asking an inconspicuous number of questions. And legally, it is difficult for America’s AI giants to argue that distillers are stealing their intellectual property. After all, in a sense, China’s copycats are merely doing to Anthropic and OpenAI what those companies did to journalists, coders, lawyers and other specialists: Feeding their public-facing outputs into a model, which then replicates their capabilities by discerning underlying patterns within the text.

Oh, and China’s giving these models away

The new Chinese models would have caused Silicon Valley enough headaches, if they merely provided stiffer competition, while demonstrating the power of distillation. 

What makes Kimi K3 and Qwen3.8 Max especially threatening to the American AI giants’ profitmaking potential, however, is that they are officially open source — meaning that the models’ parameters can be downloaded for free. (Alibaba and Moonshot have not yet released these parameters, but they say they will shortly.)

In other words, any company or hobbyist with enough computing power will soon be able to run a near-frontier Chinese model on their own hardware, modify that model to better serve a specialized purpose, and then sell access to their new version — without paying Alibiba a single yuan.

As Kimi and Qwen grow more capable, their market-share is likely to grow, at American AI giants’ expense.

For many of Anthropic and OpenAI’s potential customers, that proposition may be hard to turn down. Most businesses don’t need the world’s smartest AI, just one competent at their enterprise’s core tasks — compiling legal research, answering IT queries, writing working code, etc. A model that produces outputs 90 percent as good as Claude’s — at roughly one-sixth of the cost — will sound pretty good to many corporations.

Further, open source models aren’t just cheaper than frontier systems, but potentially more secure. If you run an AI on your firm’s own servers, then you don’t need to entrust sensitive data to Anthropic, Google, or OpenAI.

All this had led much of corporate America to embrace open-source models, even before the latest versions narrowed the capabilities gap. In a Linux Foundation survey, 63 percent of organizations reported using open-source AI systems.

And increasingly, those models are Chinese. According to Sequoia Capital, one of Silicon Valley’s premier venture capitalist firms, a majority of American AI startups now use open-source Chinese systems. As Kimi and Qwen grow more capable, their market-share is likely to grow, at American AI giants’ expense.

What’s bad for OpenAI is good (and/or catastrophic) for humanity

All this said, it is still entirely possible that OpenAI and Anthropic will justify their colossal valuations. In many highly competitive economic domains, having access to the world’s very best AI model will remain highly valuable. And America’s frontier labs still outperform all their peers. 

But it’s increasingly plausible that selling state-of-the-art AI systems will prove to be a low-margin undertaking. In a world of ubiquitous, near-frontier open source models, the AI sector’s big winners probably won’t be its top labs, but rather, its chipmakers and cloud computing providers. 

For ordinary people, a future where superintelligence is dirt cheap — and rival AI companies are constantly rising and falling, rather than consolidating into mega-corporations — would look somewhat different than the cyberpunk dystopia that the left’s been dreading. 

And not entirely in a good way. For one thing, in that reality, mitigating AI’s biggest risks would be immensely difficult. Having a handful of firms monopolize control over frontier AI systems is bad in many respects. But it does make those models easier to regulate, as the Trump administration’s decision to temporarily block Claude’s Fable in the name of cybersecurity demonstrated. 

By contrast, if recipes for ultra-powerful AI models are published all over the internet — and anyone with modest technical skills can modify them at will — then systems willing to help their users hack government bureaucracies or engineer bio-weapons are liable to proliferate.

From another angle, however, the “AI becomes almost free” scenario may look like capitalism at its finest: Retrospectively, such a development would mean that a small number of extremely rich people bankrolled the creation of an immensely useful technology, under the expectation of massive profits, only to see competition erode their returns — and disperse that tech’s benefits across a wider group of businesses and consumers. 

Granted, in the case of AI, this process might also generate a super-virus that kills us all. But hey, no system is perfect.

The comical charm of Meta’s new AI-enhanced, smart glasses is that they’re seemingly made for someone unimpressed, perhaps thoroughly, with the way they’re experiencing life. 

Being unhappy with life in its current state is, of course, intrinsically human and a problem that is as old as time. In fairy tales and folklore, the motif ends with a lesson — sometimes with the aid of magic, divine punishment, and maybe a witch or two — to be thankful for the things you have. More contemporary interpretations involve time travel and alternate realities, and perhaps some kind of Christmas theme.

Now, in 2026, we have glasses imbued with technological magic. The $300 Meta glasses are equipped with dual cameras as well as multiple microphones and tiny speakers that connect to an AI-powered app on your phone. This allows you to understand languages you can’t speak, command music to be played, perhaps even identify people you don’t know, and record anything you want and keep it, should you desire, forever. Having the ability to do these things, according to Meta, could make life a little better. 

The problem with Meta’s pitch is that the improvements the glasses promise come at a price. These super-powered spectacles run the risk of annoying or offending or creeping out the people around you. 

And worst of all, even if you’re just plainly wearing the glasses, they’re killing the vibe for everyone else.

Wear Meta glasses at your own risk

The major takeaway from Meta’s advertisements and overall promotional strategy for its glasses is that you’re supposed to wear them everywhere: Japanese restaurants, nights out at bars and clubs, on escalators (possibly to somewhere cool), backstage at concerts, at Kardashian houses, in Miami Beach, skydiving, and everywhere in between. 

What these stylish commercials and dynamic morsels of marketing do not tell you is that at all these places there will likely be people who do not want to see you wearing these glasses at all. 

“For a few weeks there was a trainer at the gym I noticed wearing glasses — black Ray-Bans — and he had never worn glasses before. It took me a few days but I realized they were Meta,” Sean, a non-Meta glasses wearer in DC, told me. Vox agreed to let Sean and other people interviewed for this article to go by their first name or pseudonym to let them speak freely about these spectacles. 

 “I almost said something to a manager, but then he stopped wearing them before I could say anything,” he added. 

Sean explained his unease. All around this country, people film in the gym all the time — to the point where it’s obnoxious and gets in everyone’s way. This trainer could just be following the trend, using the glasses (which are less cumbersome than an entire tripod setup) to film his clients’ form or creating content for a YouTube channel. But not knowing what the glasses are for, what the trainer is recording, or where he’s recording is what bothered Sean. 

“I don’t want to be in the background of videos all over the internet or on TikTok,” he said. 

There’s a little bit of social absurdity here in that we’ve been encouraged to post and have so many platforms — TikTok, Instagram, YouTube, etc. — to do so. Meta has continually emphasized that not only do these glasses allow people to share their points of view but also that everyone’s point of view is so important that it needs to be shared. At the same time, more people posting more than ever has made people aware of being a background character in someone else’s content. And it turns out that real-life, regular people are not particularly invested in how the stranger next to them in Meta glasses sees the world. 

Patrick, a writer living in New York, told me about a wedding he’d recently attended, where he saw an old friend wearing a pair. His group of friends told their pal that they wouldn’t talk to him until he took them off. 

As someone who aspires to be the kind of person you want to sit next to at a wedding, this is understandable. A wedding is theoretically two people sharing the most romantic day of their lives, but it is also a well of gossip, judgment, inside jokes, and lore that’s shared by the people who are watching said couple share the most romantic day of their lives. There’s also the possibility that, depending on the wedding, the spirit might move oneself to partake in the erotic violence known as the “chicken dance” and would not want any of that recorded. 

“One person screamed, ‘Ew the Kylie glasses, gross.’ It felt so cathartic,” Patrick said, referencing Meta glasses spokesmodel Kylie Jenner who, in her personal life, may or may not wear the glasses she advertises.  

In addition to weddings and the gym, people also told me that they don’t want to see the glasses at restaurants, not on dance floors or at parties, and definitely not in an immersive theater setting. I even spoke to someone who started a petition to ban them from bars. 

“I’m staunchly anti-photographs at good parties — the best parties in the world ban photography from the dancefloor,” said Vee, a nightlife aficionado who has owned but does not use his pair of Meta glasses anymore. “When people know there’s a camera on them, they behave differently. They start to monitor their own behavior. They become self-conscious and they start to think, Well, what will the people watching this video think I’m doing? Am I being cringe?” 

Going out partying is a completely different experience from lifting weights at the gym, and both are obviously very different from attending a wedding with friends. Yet, the critique of wearing Meta glasses at all these places is the same. No matter the vibe, Meta glasses will kill it and flatten the mood. People can’t enjoy the moment because the moment is being intrinsically changed by a person who might be recording. 

One of the rather unfortunate terms that Meta glasses have acquired is “pervert glasses.” This is largely due to the trend of pickup artists, pranksters, and yes, perverts, who are using the glasses to film others without their consent. Vee, the dance enthusiast, told me that it’s one more reason why he doesn’t believe that these specs have any place in nightlife.

“There’s a very significant community of people who trade videos that they’ve taken: candid videos of women who are in various states of undress. It’s just this kind of seedy underground,” Vee said, pointing out that Reddit had to ban entire forums dedicated to sharing nonconsensual videos and pictures of women at festivals and clubs. 

As Vee explained, people at festivals and nightlife events are often partaking in drugs and alcohol. He has no problem with that. What he does take issue with is that no one doing these things should have to think about being someone else’s content. Intoxicated people aren’t in the state of mind where they can consent to being in someone’s video or are even aware that someone’s Meta glasses have their recording light on. 

“There’s this whole kind of creeper contingent of folks who are trying to learn how to disable the recording light,” Vee said. “It’s not everybody who owns these glasses obviously, but there is a significant number of creepers who are giving the hardware a very bad name.”

Are we being too mean to pervert glasses?

The indicator light has since become a major flashpoint. The light is part of the glasses’ vibe-kill persona as it makes clear to everyone that can see that recording is taking place, and it’s especially vibrant in dimly lit places. The effect, I imagine, is like when the toys in Toy Story collapse when they see a human. That’s led to a contingent of glasses-wearers who want to disable the feature. 

While it’s understandable that someone might not want to draw even more attention to these glasses, hacking the light source is something a creeper would also do. Hence the “pervert glasses” moniker. These privacy concerns are why Meta is now rolling out an update that will disable the spectacles’ recording feature if said light is tampered with. 

“I think if you tamper with that, they should just break,” Jason, a 28-year-old Meta glasses owner, told me. “It’s the thin veil that makes it like, ‘Okay, at least you’re telling me you’re filming me.’ So the idea that if you tamper with it? No, your product should be bricked.” 

Jason bought a pair of Meta glasses in November with the idea that he would use them to film food content and post reviews. He had even picked out a name, “Sandwich Digest,” and dreamt of its success. But on his first wear, he took them to a Japanese sandwich shop and quickly realized that his future as a Meta glasses-wearing food critic was not a fun one. 

“I felt so uncomfortable,” Jason said, noting that the obviousness of his Meta glasses and their recording light spiked his self-consciousness. “But I also felt like the people I was interacting with were also uncomfortable. And I felt like they were feeling like, Okay, we both know this is weird, but I can’t say that because you’re actively filming me.” 

The strange interaction Jason described is a version of the panopticon effect, the idea that being recorded makes people alter their baseline behavior, perhaps to the point where people self-regulate even when they’re not being recorded. For Jason, there was no “real” restaurant experience to be filmed because everyone was so uncomfortable with being recorded. 

Therein is the conundrum: Even if you’re not using these glasses to be a creep, people still think you’re a creep, and you know that they think you’re a creep. 

Therein is the conundrum: Even if you’re not using these glasses to be a creep, people still think you’re a creep, and you know that they think you’re a creep. 

Since that initial encounter, Jason tells me that he’s only ever used the filming feature to capture a zipline experience he had in Mexico at the beginning of the year. He thinks he might also use them to capture hikes or excursions on an upcoming vacation, but he won’t use them in the vicinity of or  to film other people — especially since they’ve been dubbed pervert glasses.

“People’s perception of these glasses now is not what it was when I bought them in November,” he told me. “And had it been then, and if I knew what I know now, I probably wouldn’t have bought them.” 

Max, a Meta glasses wearer based in Australia, is a bit more keen on them but still shares some of Jason’s sentiment about their not-so-great reputation. Max explained to me that he has two pairs and wears them around 40 hours per week, mainly when he’s working, or walking and driving from place to place. Though he says he’s never had a negative interaction when wearing his glasses, he understands the backlash. 

“While we’re constantly being recorded when in public anyway, there’s a big difference between mass surveillance and one random weirdo recording you for their own purposes,” Max said, making the point that the indicator light, as awkward and obnoxious as it is, could be the accessory’s most important feature. 

“It’s really the only defense wearers have against the ‘pervert glasses’ remarks,” he added. “If the firmware update is successful, and the marketing around the update is widespread, I think we can slowly turn the public in their favor again.”

To be clear, many Meta glasses owners like Max pointed out that the glasses do have extremely useful features, like hands-free calling or the ability to translate foreign languages in real time. Meta also touts the glasses’ accessibility features, including those for people with reduced vision or hearing. They also offer sun protection. 

These are all ostensibly helpful gizmos, and their existence seems to indicate that Meta and its tech cohort are trying to figure out how to make these glasses as essential to our everyday lives as smartphones. Perhaps, when these features become more innovative or exciting or if the glasses become so popular, the narrative around why people buy them may change. 

But for now, the singularly intriguing thing about these specs is simultaneously their most publicly maligned feature: the filming. 

“There are probably a million things to film with them that are probably not weird,” Jason, the one-time Meta glasses food critic, told me. “But like if you’re wearing them at the beach, I would probably be side-eyeing you because you’re wearing ‘pervert glasses’ at the beach — fork found in kitchen.”

AI was supposed to make our lives better. Instead, it’s made many of us scared and angry. Communities are protesting against the building of new data centers — the warehouses of IT equipment powering the AI buildout — across the country, and increasingly they’re winning. And polling shows most Americans think AI is moving too fast.

So how did public opinion on AI curdle so quickly? Jasmine Sun, who reports on the industry from San Francisco, argues that the backlash treats AI less as a technology and more as a political project. “The debate was not about like, is ChatGPT useful to me?” Sun told me during a taping of Vox’s The Gray Area. “The debate was actually something more like, there are these big corporations and unaccountable billionaires…coming into my city, coming into my life and changing it without having any sort of democratic input?”

Filling in for Sean Illing, I talked to Sun about the rise of “AI populism,” the parallels with the Industrial Revolution, and how the backlash could crash into the 2028 presidential election. 

As always, there’s much more in the full podcast, which drops every Monday, so listen to and follow us on Apple Podcasts, Spotify, Pandora, or wherever you find podcasts.

You’ve been writing about a phenomenon you call AI populism. How would you define that? What is AI populism?

I define AI populism as a worldview where AI is not seen as an ordinary technology, but specifically as an elite political project to be resisted. I came to the term while thinking about the AI backlash and the reasons people are increasingly anti-AI — whether that’s LLM slop, whether that’s Waymos in their city, whether that’s a new data center project. One thing that occurred to me was that a lot of times the debate wasn’t about whether ChatGPT is useful to me, or whether Waymos are safer than a human driver. The debate was actually something more like: There are these big corporations and unaccountable billionaires who are coming into my city, coming into my life, and changing it without any democratic input.

When I talk to people who are opposing AI in various ways, they seem more concerned with this concentration-of-power, anti-elite dimension — which is where I take the word “populism” — rather than classic AI safety concerns, which are more about the technical characteristics that might introduce risk.

You wrote a piece that touched on some of this but went to a darker place — “AI populism’s warning shots” — and you wrote about actual shots. Sam Altman, the CEO of OpenAI, was targeted by a Molotov cocktail and a shooting within the span of a couple of days. There was an Indiana councilman who voted for a data center and woke up to gunshots at his home and a note reading “no data centers.” Why do you think of those incidents of violence as warning shots of something to come?

It was pretty scary. I’m no Sam Altman fanboy, but it’s terrifying that assassination attempts are showing up in response to people’s worries about AI. One factor is that we’ve been seeing a rising wave of political violence and support for political violence in the US, especially among young people, over the past few years — the UnitedHealthcare CEO shooting, the Charlie Kirk shooting. Increasingly, a lot of disaffected, maybe nihilistic young people are turning toward political violence as a way to express political beliefs they don’t feel they have other channels for. Or maybe that person is just unwell. But I do expect to see more of it, because my theory of political discontent is that if people feel they have institutional channels to bargain for their rights — if they feel the democratic process is working, or they’re part of a union and believe their union leader will go bargain about how automation shows up in the workplace — they’ll most likely go through those channels.

When it feels like the official channels aren’t working, opposition becomes much more diffuse and volatile. That’s part of why, in creative communities, you’ll see people witch-hunting each other over AI use. I think we’ll see more political violence against people seen as AI leaders, or as supporting AI leaders.

That’s really scary.

Yeah, I’m quite worried about it. But again, my sense is that it comes from a feeling of — what else is there to be done, when you have this level of concentration of wealth and power, and there’s no democratic input right now into how AI is regulated or built?

It’s like a jump straight from complaining at your community meeting about the data center to an act of violence.

I was talking to some friends about this. During the 20th century in the US, there was a wave of factory mechanization and automation, but unions were really strong — often when a company said, “We’re going to bring in these machines,” they’d sit down with the factory union leader and say, “Okay, you can bring in the machines, but we’re going to couple that with a wage increase,” or a 35-hour workweek, or earlier retirement. There was a channel to make a deal about how automation would show up in your workplace. That meant people were more likely to accept it as something lifting all boats. I don’t think that’s happening now — most of the industries affected by AI aren’t organized in labor unions, and the democratic channels are questionable at best.

It’s like when people have agency to be part of the transition, the process goes a lot smoother. Is there a historical analogy for a technological change that didn’t allow for input from the people involved? I’m thinking of the Luddites.

The Luddites are a good example. When the automated looms were introduced, there was a lot of violence against the looms. The book I’d really recommend here is Carl Benedikt Frey’s The Technology Trap. He’s an Oxford economist who studied a ton of historical examples — in Europe, in China, all over the world — including the Luddites and 20th-century automation. His central question was: In what contexts do workers successfully stop automation, and in what contexts do they allow it to be introduced? How does the political environment, or the balance of power between people and their leaders, change the outcome? He found that when automation was introduced alongside social welfare policies — a higher minimum wage, some form of redistribution — people were much more willing to accept it, which is fairly rational.

I want to talk about Silicon Valley’s understanding of this backlash more generally. You’re painting a pretty dark picture, and you’re right in the belly of the beast in San Francisco — I’m sure you talk to people involved with AI every day. Is there a moment when it clicked for them that this backlash is real and something they have to take seriously? Or has that happened yet?

I’ve definitely noticed a huge difference, over the past six months, in how seriously people in Silicon Valley take the AI backlash.

Like what?

People just talk about it more. I’d bring up AI populism to people last year, and they’d normally say, “It doesn’t matter — technology always introduces some discontent, people get annoyed but they get used to it, like the internet.” That was the standard reaction last year. Not anymore. I think part of the reason OpenAI and Anthropic have felt pressure to introduce economic policy proposals around job automation is that they’re seeing how worried people are. The data center moratoriums and the broader data center backlash have been surprising and meaningful in getting AI leaders to recognize they have both a messaging problem and an actual problem with the product and the technology they’re introducing.

A lot of the increasing opposition to AI in Washington has caused people to see this too. At first, Trump — as you mentioned — was very pro-AI. He and David Sacks were accelerationists; they wanted AI to go faster and to block attempts at regulation.

He was the AI czar.

“The moratoriums, the regulation fights, even the booing at graduations, the literal assassination attempts — people in Silicon Valley have become much more worried.”

He was the AI czar — he’s no longer the AI czar. But it turned out a lot of other constituencies, both on the left and the right, were pretty opposed. For example, Trump and David Sacks tried to introduce a big federal bill that would preempt all state-level AI regulation — no state could regulate AI for 10 years. They tried to sneak it into a big omnibus bill so no one would notice. But members of Congress realized it was happening, and — whether for kid-safety reasons or frontier-safety reasons — people said, Wait a second, the idea of preventing any state from regulating AI for ten years is crazy. A lot of people organized in Washington to successfully stop that preemption. I think that showed the scale and bipartisanship of a coalition that was very keen to make sure it stayed possible to regulate AI was underestimated. As a result of all this — the moratoriums, the regulation fights, even the booing at graduations, the literal assassination attempts — people in Silicon Valley have become much more worried.

China is our big competitor in the AI race, and it certainly has all the conditions for a populist pushback to AI — youth unemployment is really high, and AI technology is in some ways more advanced at taking over real-world jobs. I was watching a video about fully automated factories and a robot pharmacist. You’re one of the rare American tech reporters who gets to spend time in China, and you wrote a piece that surprised me — you found there wasn’t really a populist backlash to AI there. Why not?

I was really interested in this question, and I was finishing my New York Times piece while in China for a few weeks, talking to both AI people and non-AI people. The main reason there’s not a big populist backlash in China is that there isn’t a lot of social unrest or populist backlash against anything — the entire MO of the Chinese government, the No. 1 priority, is domestic social stability. Any whisper of protest gets shut down; that’s why they have such strong speech controls. So one factor is that China doesn’t have much of a culture of resistance in general, whether in workplaces or politically. I’m not saying no one dissents — but it has a cultural effect too, because people don’t see it as useful or as an option. When I ask family members of mine in China about AI, sometimes they’re annoyed about specific things, but fundamentally, the idea of opposing AI is seen as almost unimaginable.

The other thing about China is that if you’re middle-aged there, you’ve lived through so many political, economic, and technological revolutions in your lifetime. When I was a little kid visiting Shanghai in the mid-2000s, there were no high-speed trains — now China has some of the best high-speed rail systems in the world. Technology has always gone hand in hand with dramatic economic advancement, with being lifted out of poverty. The modernization process has been aggressive and disruptive, but it’s not something the party has offered opportunities to resist, and it’s something most Chinese people still see as an inevitability that was mostly good for most people — because incomes did increase by dramatic amounts alongside the technological change. So I think people have a similar attitude toward AI: It’s much less about “Can I stop the AI wave?” and more “How can I take advantage of the AI wave to get ahead economically?”

We were just talking about this deep pessimism about what technology can bring us here in the US. I think a lot of people look around and think: We don’t have a cure for cancer yet, but we’ve sure seen our lives get worse in a lot of ways because of technology, social media, whatever. That pessimism probably fuels the backlash to AI, the skepticism about whether it can ever deliver on its promises. And that experience just isn’t the same in China, or probably much of the rest of the world, where technological progress has been faster and more concrete in people’s lives.

My 90-year-old grandfather said he’d love an elder-care robot to help him do tasks around the house so he doesn’t have to rely on his kids — he wants more freedom and mobility. It’s seen more as a tool to help individual goals. Even with the robot factories or pharmacies — one thing that struck me visiting a robot pharmacy was that the PR people happily said, “Yep, we’re doing these robots because human workers take too many smoke breaks and bathroom breaks and take too long.”

You’d never say that in the US, but they’re probably thinking the same thing — they just don’t say it. The other thing they mentioned is that this lets the pharmacy operate 24/7, because a lot of people need medications in the middle of the night and want to order via the DoorDash equivalent. There was actually a labor shortage before — Chinese workers weren’t willing to work night shifts — so these pharmacies are offering real consumer surplus. A significant percentage of orders come in overnight, when no other pharmacy is open. And with the factories, part of the issue is that Chinese workers, especially young people, don’t want to do factory work anymore.

“I think the 2028 presidential primary and election is really where I expect AI to become a centerpiece of the conversation.”

That anecdote gets at the promise and peril of AI, and the role of the backlash movement — which I’m still wrestling with how I feel about. On the one hand, I want to live in a world where cancer gets cured, where we live in an era of abundance, where things are cheap and easy to make because factories can run all the time with machine workers who don’t require anything — I want the future we were promised, of flying cars and everything working well.

But I also don’t want to lose my job, or see humanity wiped out by an angry machine god. Because we don’t really know what’s going to happen yet, it’s hard to work out my own feelings about the pushback here in the States — what’s appropriate, and what’s holding us back from real advances in our lives.

Totally, I agree. I like Waymos — I think they’re safer, and I’d prefer a safer robot car driv[ing] me around instead of me driving. I’m not a good driver; no one should let me drive. So I wrestle with some of the same things.

To close out the conversation — let’s come back to the United States. AI populism is brewing as a political force. We’ve seen it show up in a couple of races so far, but it’s early. We’ve got the midterms, then the presidential election. How do you think it’s going to affect American politics this November, and in 2028?

My sense is that this November, it’s going to be more about state and local races where AI really shows up. I’m going to spend some time in Michigan and Wisconsin this summer touring some of the data center sites facing the most opposition — those states also have contested governor and Senate races where AI and data centers have become a core issue, so I’m interested to learn more there. I think the 2028 presidential primary and election is really where I expect AI to become a centerpiece of the conversation — especially if we start to see some of the employment impacts people are expecting. As soon as we see something like a 2 percent rise in unemployment, if that happens, I think people will be very upset, and we should expect a ton of focus on the issue.

The other thing I’ll note is political opportunism — you’re already seeing a bit of this, where politicians are likely to raise the salience of AI above where people might ordinarily care about it, because it’s become a convenient boogeyman. It polls so poorly, people are so anti-AI and anti-data-center, AI billionaires are so unsympathetic, that no matter what your policy program is, AI is a great reason to push it. I think a lot of politicians who are being clever about this are going to move AI to the center of the conversation, raising its salience to manufacture urgency for proposals they’re already excited about. That’s definitely something I’m watching for 2028.