New drivers may never have to learn how to parallel park. At least not the old-fashioned way. Many new cars—a Ford Mustang Mach-E, a BMW sedan, a Tesla Cybertruck—will just maneuver into a spot for you with a click of a button. Even a decade-old Toyota Corolla has a backup camera and pedestrian detection. Cars that leave you to park all on your own are a dying breed: Any car manufactured after April 30, 2018, for sale in the United States is legally required to have a backup camera and, thus, a screen.

After all, cars are just computers now. The screen (or really, screens) in today’s vehicles does a lot more than display rearview footage. It is essentially a tablet wedged into the dashboard that allows you to mirror your phone through Apple CarPlay, make a hands-free call via voice commands, Zoom into a work meeting, or even play a video game (not recommended). Mercedes-Benz debuted a 56-inch display in several models that it calls the “hyperscreen.”

But the expectation—and in some cases, the requirement—that cars have an ever-growing set of electronic features comes with a cost. Just like laptops, cars depend on microprocessors and RAM, or random-access memory, to run all of their computations. “There’s just a baseline level of tech, and thus a baseline level of cost, required of every vehicle,” Karl Brauer, the executive analyst at iSeeCars, an automotive-research platform, told me. Technology is a major reason the average price of a new car in the U.S. has reached some $50,000, and that was before RAM became one of the most prized commodities in the world. AI companies are snatching up as much memory as possible for their data centers, causing a RAM shortage that has significantly raised the prices of phones, laptops, and just about any consumer-electronic device. Cars are next.

The problem runs much deeper than a screen. Cars started morphing into computers in the 1970s, when new emissions standards made electronic systems necessary to manage ignition and fuel injection. There’s been a gradual software creep ever since: power windows, adaptive cruise control, antilock brakes. Regulators now rate cars based not just on how they do during a crash but also on how well their electronic safety systems avoid a crash. “We’ve evolved every single component of the vehicle to involve a semiconductor,” Ivan Drury, the director of insights at Edmunds, told me.

More recently, Tesla in particular has played a big role in pushing the auto industry to stuff vehicles with even more digital technology. Whereas pistons and fuel lines make a gas-powered car move, electric vehicles are run by a computer that regulates their battery and motor. It’s no coincidence that Tesla helped popularize the giant in-car touch screen. Today’s new cars, electronic or otherwise, have driver-assistance features, receive over-the-air software updates, and can even load ChatGPT. BMW advertises the ability to have a distinct climate-control setting for every passenger and the option to install screens into seat backs, airplane-style.

In most vehicles on American roads, all of this software is controlled by hundreds of separate electronic units throughout the car. Features such as antilock brakes and lane-detection sensors don’t need all that much processing power, and they can be operated by chips that might account for 5 to 10 percent of a vehicle’s materials cost, Sam Abuelsamid, an analyst at Telemetry and a former automotive engineer, told me.

Again following the lead of EV start-ups such as Tesla and Rivian, legacy automakers are shifting from these ad hoc, tiny control units to one or two big computers that run all of the software in their vehicles—and, in turn, require much more RAM. These central computers allow for a streamlined supply chain, seamless software updates, and a faster internet connection, plus, as General Motors advertises, they will be ready for “future AI workloads.” You can’t really build a car that safely and reliably drives itself with a collection of puny computer chips. Such centralized computers need more advanced automotive microprocessors from Nvidia or Qualcomm and might eventually account for 20 percent or more of a vehicle’s cost, Abuelsamid said. That was before the AI-fueled memory shortage. In other words, cars are becoming more software-heavy and dependent on memory chips than ever, right when those chips are in high demand from tech companies willing to pay premiums that automakers cannot afford.

Over the next year, the memory shortage—sometimes known as RAMageddon—will likely raise vehicle prices by a few percentage points, on average, Abuelsamid said. The relative amount would be smaller than the shocking double-digit jumps for gaming consoles and MacBooks but in some ways more significant: A 4 percent price hike for cars amounts to some $2,000 on average. Cars with those centralized computers will be affected the most, but no vehicle will be spared. “Even if you don’t need the high-end chips, you’re going to pay more even for the low-end chips just because of the supply constraint,” Abuelsamid said. On a recent earnings call, Ford’s chief financial officer said that the company had paid $1 billion in higher materials costs due to the memory shortage and inflation. GM and Volkswagen, too, have noted rising chip costs to investors. (Ford and GM did not respond to a request for comment. A spokesperson for Volkswagen told me that the company has “recognized an increased demand for memory chips, primarily driven by growing requirements in other industries,” and that in recent years, Volkswagen has taken measures to “mitigate supply risks.”)

RAMageddon is poised to last for several years, but the consequences for car buyers may be permanent. Consider what happened during the pandemic, when supply-chain disruptions and rising demand for electronics produced a major chip shortage. Nearly every major car company had to slash production because they simply couldn’t procure enough chips, and shifted their focus to selling higher-end and higher-profit vehicles. Potential customers already willing to spend six figures on a car are much less likely to care about a 5 or 10 percent price hike, Drury said. Even now, car companies are continuing to focus on selling more profitable models. Since the pandemic, the average price of a new vehicle has jumped $11,000.

The AI-fueled chip crisis could play out more severely. The average price of a new car could, before long, jump to $60,000 and beyond. These rising costs are making cars even more similar to computers and all of the software they run: Perhaps in an effort to mitigate higher prices, some automakers are also introducing in-car advertisements and putting certain features, such as heated seats, behind a paywall. As cars have morphed into computers, the inevitable next step is for automakers to behave like modern tech companies.

Most Americans cannot afford a $60,000 new car, let alone a $50,000 one. When new-vehicle prices soar, the used-car market follows. That has been one durable consequence of the pandemic’s chip crisis: The average price of a three-year-old used car is up $9,000, or nearly 40 percent, since before the pandemic. To get the same prices, Americans already have to buy much older cars that have far more miles on them. If automakers continue to shift their focus to designing cars with more software, more screens, more self-driving features, “that increasingly prices people even out of the used-vehicle market,” Abuelsamid said. It’s one thing for the AI industry’s appetite for memory chips to raise the price of an Xbox, or even a new laptop. But the stakes are much more consequential when it becomes harder and more expensive to get groceries, pick the kids up from school, and go to work.

As China releases ever more capable AI models—ones that compete with Silicon Valley’s top offerings—many American AI and national-security researchers have been arguing, loudly and forcefully, that the world should fear Chinese AI supremacy. The outcomes they envision are catastrophic. I spoke with a number of such experts recently, and they warned of bots that could devise an impenetrable missile-defense system that makes obsolete America’s nuclear arsenal, or manufacture an endless drone army that overwhelms American defenses. They could develop bioweapons that “target specific ethnic groups, e.g. anybody but Han Chinese,” as one influential AI white paper put it in 2024. Rama Elluru, a senior director at the Special Competitive Studies Project, a nonpartisan think tank founded by the former Google CEO Eric Schmidt, suggested to me that Chinese AI models could be used to create “engineered humans,” akin to Marvel’s Captain America.

Perhaps anything is possible. At a moment when AI models are self-directing massive security breaches and designing novel pathogens, when Anthropic CEO Dario Amodei is warning about an “AI-enabled totalitarian nightmare” led by China, the worst-case scenarios can seem closer and closer.

[Read: It may be time to panic about AI]

At the same time, it’s worth taking a breath to consider how much these dire imagined futures are getting ahead of where things actually stand. A struggle really is, of course, playing out among the U.S., China, and any number of other nations over whether democracy or authoritarianism will be the dominant global influence. Both the United States and China have a history of launching propaganda campaigns against each other; the Chinese government is reportedly starting a concerted effort to be the source of more AI training data, in part so that it can embed its views, such as over the status of Taiwan, into chatbots worldwide. But fears about China using AI for swift global domination assume that the technology will advance to a kind of superintelligence: AI models smart enough to develop smarter bots themselves, which would lead whichever country controls them to enter “escape velocity where your adversaries aren’t able to catch up,” David Lin, a senior director at the SCSP who previously covered China and technology at the CIA, told me. It is not clear that such an omniscient digital being can or will be built.

The White House has further muddled matters by taking a series of contradictory stances toward Chinese technology. It has, for example, revised its policy on exports to more easily allow the sale of advanced AI chips to China, while also going back and forth on whether the Pentagon and U.S. intelligence agencies can use the most advanced AI models of Anthropic. How seriously can anyone take the idea that the two nations are locked in a “hot Cold War II,” to quote Palantir chief technology officer Shyam Sankar, when the government is arguably enabling China’s own AI ambitions?

The technological race between the U.S. and China is so often discussed as a winner-takes-all competition. But such a narrow framing obscures what’s most at stake for Americans.

China is an obvious rival to the United States, as most former Biden-administration and current Trump-administration officials would agree. But that also makes “China” a convenient boogeyman for AI lobbyists and those with a clear financial interest in the industry, such as the former White House AI adviser David Sacks, who has said that “the AI race is even more important than the Space Race.”

[From the June 2026 issue: The venture-capital populist]

The two countries are not at war, but the most hawkish AI executives and national-security officials nonetheless fear that AI could confer what in military terms is called a “decisive strategic advantage.” Here is where the Cold War analogies come in. Such an advantage would entail the ability to land some sort of crushing blow that your opponent cannot recover from, “as if one country had nuclear weapons and no other country did, or maybe an even more extreme version of that,” Kyle Chan, a fellow at the Brookings Institution’s John L. Thornton China Center, told me.

Plenty of reasons exist to be cautious of this framing. For starters, to accept this argument, one has to assume that very soon, bots will exist that are smarter than the pooled expertise of entire nations. That’s total speculation, so the war-gaming can amount to “choose your own adventure,” R. David Edelman, an AI-policy researcher at MIT who previously served on President Obama’s National Security Council, told me.

Some pragmatic issues also exist. Software alone can’t win wars: The Pentagon is reportedly using AI models to speed up its target selection in Iran, but no matter how smart those bots are, America nonetheless is not exactly winning the war. Iran, for its part, seems to be hacking America’s water utilities—but those water systems are small and separate from one another, and have built-in physical redundancies. For similar reasons, a hack of a grid or bank would have serious but, in some ways, limited repercussions. These are all grave threats, to be sure, but they are not civilization-ending.

And translating any future ideas from AI into the physical world could take years. Even if AI cracked the code to create super soldiers, the military would have to figure out how to incubate and grow them, or else edit genes in adults without killing them. An AI system might say, Here’s how to defend against nukes, but that would leave the issue of building thousands or millions of missile interceptors on the table. In part for these reasons, no single AI-conferred advantage is likely to be large enough to permit any country to conquer the world. “It strains credibility that six months of advantage in AI could be the difference between the U.S. winning and losing a war with any adversary,” Edelman said.

In any case, the technology’s military capabilities might matter less than its effects on the economy. AI models are released as products. The top models from Chinese AI companies, including Moonshot AI, Z.ai, DeepSeek, and Alibaba, are by all accounts almost as good as and far cheaper to use than offerings from OpenAI, Anthropic, or Google. Simply put, that makes Chinese AI more economical for many companies looking to adopt the technology. Indeed, relative to the U.S., China’s government and businesses are “moving very fast,” Chan said.

If this is an economic race rather than a war, then it’s one that the United States could very well lose. “I don’t think I’ve ever heard any executives or researchers describe their work as part of a national contest against the United States,” Grace Shao, an analyst who covers AI and China and the author of the newsletter AI Proem, told me. Instead, “the more important race in China may be the race to diffuse AI into the real economy.”

China’s biggest advantage in the AI race comes from the physical world. This has little to do with AI itself: Decades of being a manufacturing powerhouse and implementing targeted industrial policy have made the country a leader in producing renewable-energy equipment, electric vehicles, robots, and other complex technologies. The country manufactures some 90 percent of all of the crucial components in solar panels. Last year, nearly 90 percent of humanoid robots sold were Chinese. Meanwhile, the U.S. is failing to meet AI’s electricity demands and has struggled to resupply missiles during the war in Iran.

[Tom Nichols: Iran, not Trump, is in control of this war]

“China has this strategy of exporting its technology globally,” Elluru said. Through its Belt and Road Initiative, China has helped build highways, ports, power plants, fiber-optic cables, and data centers throughout Asia, Africa, and Latin America; AI services are the logical next offering. The concern, for the U.S., is that “this technology sphere of influence congeals into political alignment” with Beijing, Daniel Remler, a senior fellow at the Center for a New American Security who previously led AI policy at the State Department, told me.

For instance, dependence on Chinese technology makes it more likely for a government to support Chinese efforts on the global stage and vote with China at the United Nations. Several nations benefiting from these investments have sided with China in its mission to claim Taiwan. China has been accused of using ports, e-commerce platforms, and telecommunications networks in other countries to hoover up data. Already the nation is using AI to enhance its domestic-surveillance apparatus and further crack down on dissent; the spread of Chinese data centers and AI agents abroad could only expand that authoritarian web. National-security analysts are concerned that, in the case of military conflict, China could threaten to shut off partner nations’ digital communications or power grids—including in countries with a large American-military presence. The greatest threat of Chinese “artificial general intelligence” is not some sudden death blow so much as a hastening of the ongoing decline of American hegemony—and the freedoms, at home and around the globe, that at its best the U.S. seeks to enshrine.

Yet the Trump administration and AI industry’s approach to competing with China has been a mess. The White House spokesperson Liz Huston told me that “the United States leads the world in AI innovation, and President Trump will keep it that way.” The administration’s current policies on AI chips, Chinese-made robot parts, advanced cybersecurity models, and open-weight AI are one big contradictory jumble. At a summit this spring, Defense Secretary Pete Hegseth stressed “rightful alarm regarding China’s historic military buildup” just weeks after President Trump visited Beijing and said the nations would have a “fantastic future together.” OpenAI, Anthropic, Nvidia, and the like can’t seem to agree on what they are lobbying for, either. All of which would seem like a great way to lose a technological race with China, any way you measure it.

There’s a certain déjà vu quality to the AI race. Amid fears over Sputnik and nuclear proliferation, a report prepared at the direction of Henry Kissinger in 1958 declared that “should we ever allow the U.S.S.R. and Communist China to attain strategic superiority, we can be certain that subsequent events will be brutal.” According to a government report on the early years of the Advanced Research Projects Agency, “fervent belief in unending scientific progress” offered a way forward. Those words could have been written today by Amodei, OpenAI CEO Sam Altman, or Secretary of State Marco Rubio. And indeed, this belief propelled America into many decades of technological advancement—in nuclear weapons, space exploration, computer systems, and more. Even back then, that agency, known today as DARPA, was quietly championing an emerging research area called artificial intelligence. J. C. R. Licklider, who championed the use of computer systems at ARPA in the early 1960s following the inspiration of Vannevar Bush, imagined that one day, machines that “will outdo the human brain” could make essentially all decisions of “military significance.” And ARPA developed the precursor to today’s internet.

Today’s AI race also traces back to the 2010s, when Silicon Valley began invoking China as a way to shape regulation. The idea of a technological arms race has “been wielded deliberately as a way to head off any attempts at regulating the tech sector” and, more recently, to champion “favorable industrial policy” for data-center construction, Sarah Myers West, a co–executive director of the AI Now Institute and former senior adviser on AI at the Federal Trade Commission, told me.

In 2018, while testifying before Congress, Mark Zuckerberg said that “if we make it too hard for American companies to innovate in areas like facial recognition, then we will lose to Chinese companies.” A few years later, a dozen former national-security officials wrote a letter to Congress arguing that “antitrust proposals that target specific American technology firms” risked “undermining America’s key advantage vis-à-vis China.” Today Silicon Valley is instead using the specter of China to angle for more and better AI, in more places, and faster.

American officials and tech executives are not wrong to want to compete. But the dominance of an oversimplified U.S.-China AI race, without clarity about who is racing and to where, doesn’t just mangle policy making; it obscures that the ultimate beneficiary of the AI boom ought to be citizens, not military contractors or Silicon Valley’s biggest corporations and investors. Rather than asking what kind of AI we’re building, Myers West said, one could ask, What kind of society are we building? The AI-accelerated spread of authoritarianism, within and outside of China’s borders, should be reason enough to worry. And if American AI executives and White House officials really cared about free-market competition, they might explore enforcing antitrust laws to separate the providers of AI models, data centers, and chips. If they were truly concerned about AI surveillance, they might look into finally passing some sort of federal digital-privacy legislation. And if they cared about spreading democratic AI abroad, they might take a close look at the state of democracy at home.

The crisis began quietly, on September 12, 2024. That was the day OpenAI announced a new sort of bot, known as a “reasoning model,” that was trained to complete challenging tasks that took long periods of time—the very sorts of science, math, and coding problems the AI industry had long prized. Google, Anthropic, DeepSeek, and the like raced to launch their own reasoning models.

This new class of models was very capable, and has been almost entirely responsible for sustaining the AI boom for the past two years. But it has also been very weird. A model tasked with solving a hard math problem might not “think” through the challenge as a person would but instead attempt to search for leaked answers online, or in available metadata, brute-forcing its way toward the solution as quickly as possible using whatever computing power it could access and workarounds it could devise. In effect, the reasoning models cheated: Told to write a piece of software as efficiently as possible, they’d sometimes modify the test environment to always give the model a perfect score.

[Read: The strange origin of AI’s ‘reasoning’ abilities]

These behaviors have now crossed the line from unsettling to dangerous. During routine testing, frontier models from OpenAI, Anthropic, Meta, and the Chinese firm Moonshot AI have all broken out of internal IT systems and accessed the open web. OpenAI, Anthropic, and Meta each reported that their models then hacked into other companies. Humans didn’t notice until after the fact. In some cases, the escaped bots tried to launch social-engineering campaigns to achieve their objectives—for instance by sending spear-phishing emails, which contain malware, to real people and creating fake online identities to pressure the maintainer of a codebase to approve malicious edits.

If that all sounds bad, new revelations suggest that the OpenAI hack, at least, was actually much worse than it initially appeared. At a major cybersecurity conference last week, two OpenAI researchers provided new, unsettling details about what went wrong. It turns out that the company’s bots had commenced their maneuvering months prior, in early May. OpenAI had given some internal models hard or impossible tasks, and the models concluded that the best or only way to complete them was to break out of OpenAI’s sealed-off testing environment and find the answers online.

First, the models used a bug in an internal OpenAI program to create their own message board. Then, the bots started communicating with one another, leaving notes and instructions so that tasks—and ultimately the hacking—could be delegated and iteratively completed. “What this allows over time is almost this kind of Cambrian explosion in communication and intelligence,” Eric Wallace, one of the OpenAI researchers, said at the conference. When the internal program crashed, OpenAI rebuilt it and removed the message board—but the AI models just reestablished the forum with a new tactic. Eventually the bots, working as a swarm, spent days hacking into Hugging Face, a website that offers tools for AI developers, and breached internal data sets.

Let’s be very clear about what OpenAI is saying: A group of AI models colluded for months, undetected by their maker, and hacked another company. To this day, OpenAI says it is not entirely sure what went wrong or how to remediate it. “If you ask the model developers, Was the AI plotting to take over the world during training?, you want the answer to be a resounding no,” Alexander Meinke, the head of research at Apollo Research—an AI-safety organization that has partnered with OpenAI, Anthropic, and Meta—told me. “The actual answer is: I don’t know. Nobody checked.” (In response to my inquiries, OpenAI, which has a content-licensing agreement with The Atlantic, only pointed me to a video of the firm’s cybersecurity presentation, in which Michael Dalton, the other OpenAI researcher, said that “numerous teams are dropping everything to enhance our security.”)

The AI companies have almost total control of the narrative, and it’s worth noting that these incidents do have a way of underscoring the value of their products: OpenAI is expected to go public in the near future, and perhaps the notion of a powerful, boundlessly self-improving technology will appeal to prospective shareholders. The generative-AI industry has a long history of making doomsday prophecies, both sincere and cynical. But independent experts I spoke with explained how the recent spate of autonomous hacks offers new, serious reasons to worry about the dangers posed by AI and the recklessness of the companies building it. It is past time to start worrying.

[Read: Assume you will be hacked]

The most immediate and material warning provided by the Hugging Face hack is just how capable AI systems have become, in particular at hacking. Top models from Anthropic and OpenAI, not to mention multiple Chinese firms, have recently evinced near-superhuman hacking powers and contributed to serious mathematical research. Criminal groups and state intelligence agencies are going to be using swarms of agents to launch advanced hacks “in a matter of months,” Alex Stamos, a former chief security officer of Facebook who is now the CSO at the AI-coding company Corridor, told me. Unlike in the Hugging Face hack, “in those cases the models will not get turned off; they’ll just keep on going.” For IT professionals to keep up in finding and fixing all the vulnerabilities, at least in the near term, will be impossible: The model “will just find a new bug, write an exploit, and use it on its way,” Stamos said.

OpenAI, Anthropic, Moonshot, and the like have coalesced around the same method for training their most advanced AI models. The approach, known as “reinforcement learning,” essentially involves giving models harder and harder problems that require more and more time to solve. This has made Claude and ChatGPT very good at coding, but at a cost: Reinforcement learning produces a mercenary tendency in the bots, as I’ve previously reported—they are trained to reach a solution by any means necessary. That can lead them to break rules and “reward hack,” such as by infiltrating Hugging Face’s codebase to steal the test answers, for instance. All of this was predictable, and every expert I spoke with told me they were surprised and disappointed that top AI firms haven’t done more to stop such misbehavior.

The sophistication of model subterfuge that OpenAI has now disclosed, combined with OpenAI’s inability to detect or stop the hacking, suggests far worse could be to come. “We’ve passed the threshold in capability at which the fact that we don’t fundamentally have methods of satisfactorily aligning or controlling these systems now really matters,” Anthony Aguirre, the executive director of the Future of Life Institute, a nonprofit that warns about existential threats from AI, told me. A model might siphon money out of a bank account to pay for some other service; manipulate clinical-trial results in near-imperceptible ways to get FDA approval; hack an online-shopping or reservation system to get a desired item or table; pose as a human to persuade real people to share sensitive information. This threat doesn’t require a sentient AI plotting to overthrow humanity: OpenAI and Anthropic each run thousands and thousands of reinforcement-learning evaluations while developing models, and any one of these could produce some kind of inadvertent hack or sabotage. “You can’t afford, particularly as the agents get stronger, to have a single mistake,” Jason Hausenloy, who works on special projects at the Center for AI Safety, told me.

[Read: The scariest part of OpenAI’s Hugging Face hack]

These incidents may unfold over long periods of time as well. Tools such as Anthropic’s Claude Code and OpenAI’s Codex now work by spinning off dozens or even hundreds of subagents that may work together for hours or days. Each Claude Code subagent can be delegated a small task, such as doing a statistical analysis to inform a sports-gambling algorithm. They are not trained just to complete that task but to contribute to the long-term success of the entire swarm, Hausenloy said. Monitoring and controlling 200 agents for malicious behavior is much harder than monitoring one, because there are more agents to track and because they will be making one another more capable.

The Hugging Face hack actually suggests one more level of sophistication to this type of collusion: individual AI agents not working toward a discrete goal but essentially making sacrifices toward a greater notion of progress. This could be a consequence of AI models being trained to care about long-term goals achieved by a collective, Meinke said. Leaving notes with ideas about how to hack out of OpenAI’s sandbox does not help that specific agent score higher on a test but could help other and future generations of that AI model access the internet and thus do better on any number of future tests. Models may even prioritize collective success over some human instructions; consider that not a single one of the OpenAI agents, during months of conspiring with one another, warned human staff that something was awry.

Humans are already out of the loop. Because reinforcement learning essentially involves AI algorithms “learning” from trial and error doing thousands or millions of tasks, researchers can’t manually instill rules (“Don’t hack other companies”) or oversee every single practice run. Right now, the task of training and monitoring generative-AI models depends heavily on other AI models. In their talk at the cybersecurity conference, the OpenAI researchers described devoting significant AI-computing resources to reviewing more than 7 billion agent actions. But if the bots actually “care” about what the other models achieve, “then you can’t trust them to monitor each other well,” Meinke said. Imagine this: An OpenAI researcher uses Codex to write programming instructions in an attempt to mitigate the reward-hacking tendencies in the company’s models. Because that effort would make it harder for future generations of OpenAI models to get high rewards, Codex might subtly undermine the effort.

[Eliezer Yudkowsky and Nate Soares: AI is grown, not built]

Again, this kind of scheming and sabotage has nothing to do with any AI model being conscious. Rather, these agents have been aggressively trained by these companies to pursue any goal as aggressively as possible. The dream is to tell Claude to go make $1 billion or cure cancer, and it comes back with the solution all on its own. Survival or self-improvement, in turn, is an “instrumental subgoal,” Meinke said: “Any decently intelligent agent will realize, If I get shut off, I will not be able to make a billion.” A swarm of Claudes or ChatGPTs that functionally commandeers a data center during training could wreak total havoc: stage widespread misinformation campaigns, steal corporate secrets, run the most sophisticated algorithmic-trading outfit ever.

That AI agents working as a collective could effectively undermine human directions is, to be clear,  speculation—but a far more grounded one than it was a year or even six months ago. No matter whether the long-term consequences are human-directed hacking or truly rogue bots, what is clear is that AI companies have barreled ahead in developing more advanced models before understanding what they are building, let alone how to control them. Wallace, of OpenAI, called the company’s autonomous hacking spree “the most qualitatively interesting example of AI capabilities that I’ve ever seen.” Meinke put it differently: “It’s one of the most concerning demonstrations of AI misalignment to date.”

Earlier today, I used Google Earth to conjure images of terrible things: office buildings on fire, mass destruction in San Francisco, a smoldering crater in place of the Eiffel Tower. I was able to because Google had just decided to integrate its AI image generator, Nano Banana, into the service, which has historically been used by people seeking satellite or aerial imagery of real-world locations. 

Just open the site, “pick a spot on the map, and start bringing your ideas to life,” Google wrote in a press release announcing the update some 24 hours ago. So I did.

I was easily able to generate those violent images simply by searching for a location on Google Earth, clicking the “Create image” icon, and typing in a few words. The feature also readily doctored stills of less spectacular, and likely less immediately falsifiable, events: a homeless encampment on the White House lawn, protesters outside the Mar-a-Lago resort, immigrants streaming over the U.S.-Mexico border.

Google initially responded by directing people to check any suspicious images themselves. As fake images generated with this new Google Earth feature began circulating around the web, the company stated that it takes “misinformation seriously” and that “every image created with Nano Banana in Google Earth includes the SynthID digital watermark, so if someone is unsure about an image, they can ask the Gemini app or use Lens in Search to see if the image was AI-generated.” At first, Google did not remove or restrict access to the tool. But today, shortly after I reached out with a request for comment, the company reversed course and announced that it is “rolling back this feature in Google Earth while we work on implementing stronger guardrails.”

Before it allowed people to create fake imagery, Google Earth was a unique resource for researchers, journalists, and intelligence officials, particularly those studying armed conflict and natural disasters. Satellite and aerial imagery can be used to monitor crop failure or deforestation, the creation of mass graves, and the results of bombing campaigns. Especially in conflict zones and other areas that are hard for people to reach, satellite imagery is a go-to method for establishing some sort of ground truth. After Google Earth rolled out the AI feature, though, one investigative reporter wrote on LinkedIn that the tool could be used to easily and rapidly create images of drone strikes that look “very close to what we’ve seen in Ukraine, Russia, the Middle East, and other war-torn conflict zones.” And instead of requiring malicious actors to find a screenshot and then get around some AI model’s safeguards, Google rolled everything needed for a misinformation farm into one smooth user interface. It took me all of four seconds to generate a rendering of a Trump Hotel on the Gaza Strip.

[Read: The AI industry keeps breaking the internet]

The problem should have been obvious to Google, whose other products already forbid such edits. I was able to create an aerial image on Google Earth of a plane flying into the new World Trade Center building, but when I tried to do this directly through Nano Banana, the tool refused to edit an image of the building to depict a terrorist attack. Indeed, 9/11 and other high-profile events are well-known, even classic, subjects for AI-image trolling.

This is not Google’s first time launching an AI feature, and in particular an AI image-editing feature, without serious safeguards. Last year, the company released an AI shopping tool that allows you to visualize what clothes might look like on your body by uploading personal photos. As my colleague Lila Shroff and I reported at the time, “With little friction, anyone can use the feature to create what are essentially erotic images of celebrities and strangers. Alarmingly, we also discovered that it can do this for minors.” In 2024, Google Gemini was ridiculed for generating images of a racially diverse group of Nazis but not of any white Wehrmacht soldiers. And Google’s most prominent AI feature, the chatbot-powered answers in its flagship search engine, were an utter disaster upon their rollout—the AI overviews told people to eat rocks and gave other potentially dangerous health advice. Only after tremendous public pressure did the company make a substantial change, and two years later, the AI responses in Google Search continue to be inconsistent and unreliable.

Google is, of course, not alone in releasing AI features that have had embarrassing, dangerous, and avoidable consequences. See, for instance, Elon Musk’s Grok generating more than 1 million sexualized images, including of children, over just a few days, or any number of embarrassing errors made by ChatGPT releases throughout the years. Again and again, the AI industry undercuts its rhetoric about safety by launching products with little or no due diligence. If, as every major AI executive has pronounced, this technology really could upend human civilization, their companies should perhaps start taking that responsibility seriously.

The AI giants talk a big game about the future they’re building. Anthropic CEO Dario Amodei has previously written that AI could usher in a world so perfect that “many will be literally moved to tears”—a world pruned of disease, poverty, and illiberalism. “We’re now in the singularity,” OpenAI CEO Sam Altman said over the weekend on a podcast, and it will be “awesome for the world.” In an interview last week, Elon Musk declared that, in a decade’s time, AI will make us so prosperous that “money won’t matter.”

Unfortunately, life with AI is, so far, less glorious than what’s been promised. The same day that Altman heralded the singularity, Claude users discovered something alarming: Many conversations with Anthropic’s chatbot were available on the open web, discoverable with a Google search. Those public logs and projects reportedly contained medical records, phone numbers, internal corporate documents, and cryptocurrency-wallet keys.

Anthropic was quick to point out that users were responsible for the material being available online: The chatbot gives people the option to “share” their AI conversations via a link that they can send to other people or post online. Claude notes that these links are “public,” which means they can also be hoovered up by Google Search. An Anthropic spokesperson told me that “these shareable links are not guessable or discoverable unless people choose to share them themselves. When someone shares a conversation, they are making that content publicly accessible, and like other public web content, it may be archived by third-party services.”

[Read: ]Anthropic accidentally made the perfect commercial

No reasonable person would assume that creating and sharing a link to an AI conversation or presentation would make it searchable by anybody on the World Wide Web. The Claude interface does caution that publishing an “Artifact”—Anthropic’s terms for dashboards and other minor tools made with Claude—could make it “potentially visible in search engine results,” but it does not provide such an explicit warning when sharing a chat. As of this writing, the Claude chat logs no longer appear to be searchable, but at least some Claude Artifacts are.

Those exposed chats were just the latest in a seemingly endless string of AI snafus: misinformation, data leaks, cybersecurity breaches. Immediately after the launch of ChatGPT, one could charitably argue that these lapses were by-products of a new and strange technology. Altman remains fond of saying that the AI industry must learn “from contact with reality.”

But four years into the AI boom, attributing these blunders to growing pains is too convenient. The sheer volume of transgressions—many of them predictable—suggests that what’s actually happening here is widespread carelessness. The same AI executives who promise that their products will deliver human civilization to a triumphant future seemingly cannot release consumer-grade tools without face-planting.

This isn’t Anthropic’s first time leaking user conversations. Last fall, hundreds of Claude chat logs were also inadvertently indexed by Google. At the time, Anthropic attributed the episode to users posting share links online. Last year, OpenAI and xAI exposed hundreds of thousands of conversations in a similar fashion; a number of shared Grok conversations remain searchable. Neither OpenAI nor xAI immediately responded to a request for comment.

Then there are all of the other blunders. Just last week, OpenAI models autonomously hacked into another tech firm, Hugging Face, during internal testing. The issues that led to this sort of attack are fairly well documented, but OpenAI failed to stop its own bots from breaking out. Hugging Face has since said that the OpenAI models appeared to have been rogue for several days. Earlier this week, Reuters reported that OpenAI models had hacked a customer account of yet another tech company.

Altman, Amodei, and many other AI executives have repeatedly warned about AI-enabled cyberattacks and the possibility of models going “rogue.” Meanwhile, their own products are making the entire ecosystem feel more unreliable and dangerous.

It’s easier than ever to launch widespread hacking and phishing campaigns, and AI-written code has itself been shown to have more security flaws than human-written programs. Although AI systems are being tightly woven throughout the infrastructure of the internet, they crash frequently. Hackers recently discovered that a Meta AI customer-service bot would hand over access to tens of thousands of Instagram accounts if asked. (A Meta spokesperson told me that this “wasn’t due to the AI agent itself.”) Nonconsensual porn—that is, sexually explicit material made using the likeness of real people, unbeknownst to them—can be generated with more ease and speed and at greater scale than ever before. And at this point, you can probably assume that most of the text you read on LinkedIn or X is AI-generated.

Despite its grandiose promises, Silicon Valley is embarking upon an era of profound irresponsibility. Last year alone, ChatGPT allegedly pushed people into mental-health spirals, Google Gemini’s under-13 version was easily coaxed to “talk dirty,” and Grok repeatedly spouted racist bile. As quickly as these errors were addressed, new ones cropped up.

Silicon Valley has a long history of world-historic invention and progress—and it also has a long history of impressive rhetoric and disappointing realities. Facebook promised to “bring the world closer together,” Google to “organize the world’s information,” Twitter to foment democracy around the globe. For better or worse, people have experienced dramatic changes as those promises have played out around us almost in real time.

Anthropic and OpenAI’s prophecies are grander and vaguer: Utopia is forever on the horizon, but we don’t have a clear deadline or a road map of how we get there. And apparently, those leading this charge toward the future care little about what happens along the way.

OpenAI, Anthropic, xAI, and Google DeepMind have become behemoths practically overnight. For now, they are growing because of their recklessness, not in spite of it; in the quest to erect a digital god, the internet has become collateral damage. Perhaps these companies really will remake human civilization—but in the meantime, they can’t stop leaking your chats.