Yet another update on the running AI discussion.

Here’s an interview Ezra Klein did with a guy named David Robinson. And that interview is about a piece Robinson, a member of the Open AI safety and safety documentation team, wrote in The Atlantic announcing his resignation from OpenAI — a decision and announcement he is self-aware enough to note has become something of a cliche. I found the interview interesting and edifying, largely on the front I mentioned earlier about finding reliable narrators.

Robinson isn’t a data scientist or a programmer. His role at OpenAI was essentially being the conduit between the teams building the LLMs and the knowledgable and interested public, specifically the people who write the documentation — what does this model do, how does it work, what are the possible safety risks. As is usually the case in any highly technical space, the people doing the work aren’t going to be writers and they’re usually not going to speak a language outsiders can necessarily understand. So you have people who are close enough to the work and the workers to understand what is happening and also able to speak human so they can communicate it to everyone else. Robinson led the team that served in that role.

For years, and especially in the last month or so, we’ve been collectively inundated with an AI industry which, uncannily and bizarrely, tells us simultaneously that AI is essential, inevitable and runs a significant chance of exterminating all of us in as few as two or three years. If you’ve dipped into this debate at all, you’ve quickly come upon discussions of so-called “rationalists” or “long-termists” or supporters of “effective altruism” who are simultaneously ubiquitous among those developing AI and in the AI safety (i.e., the folks sounding the alarm) community as well. (For the sake of simplicity and brevity, I’ll refer to these folks as the “REL nexus”: rationalists, effective altruists and long-termists. This is my coinage since I don’t know a better one.) I’ve spent a fair amount of time familiarizing myself with this world and how it relates to current debates about AI. So I wanted to share my basic impressions.

We’ve been following this conversation here in The Editors’ Blog about what AI is, what it can do and whether it can possibly live up to its hype. It’s worth considering the question from what we might loosely call not only the supply side but also the demand side. From a very broad perspective, the history of the last three decades has been one defined by the fact that you have too much capital chasing too few productive investments. That’s the driver of the recent history of cycles of booms, bubbles, over-investment and busts: the Asian Financial Crisis, the Dot Com Bust, the Global Financial Crisis, perhaps soon the AI Bubble Collapse, along with many smaller of regional ones.

As I noted a week ago, we can’t know whether the current polls will be predictive of the November election results. But the polls themselves are at least speaking with a clear voice. I want to note two things that appear to be moving in unison: consumer confidence and the Republican position on the national generic ballot.

Here’s the trend line from Nate Silver’s Silver Bulletin …

We’re now flooded with articles about the threat of AI. Here’s another one from the Times op-ed page by a tech journalist, Stephen Witt, who’s written a history of Nvidia, the chip maker powering a lot of the AI boom. Witt describes a few possible approaches to legal intervention. Interestingly, he points to Sen. Bernie Sanders (I-VT) and Rep. Greg Casar (D-TX) as two legislators pitching new laws to get in control of the situation. Casar has a quote that captures it all and illustrates how absent “externalities” are to anything about how we regulate tech: “If the A.I. companies understood that their products have to be safe before they can advance, then they will make the kinds of investments that we need.” Of course, having two left-wing legislators from the party which isn’t even in power as the two guys on the case doesn’t inspire a lot of confidence that anything is going to happen any time soon.

You’ve probably seen these reports of an employee of the AI company Anthropic who quit his job and issued a warning on X that suggested AI could, in next few years, escape control and lead to the extinction of humanity. These sound like not just wild but science fiction-type claims. And they’ve lurked around frontier AI labs for years. It’s one of the many weirdnesses of the AI world and those who lead it. The basic pitch: AI is awesome. We need to built it as quickly as possible. Also it might lead to human extinction. Quite an attractive offer! There’s always been a strong sense among many observers that these claims or suggestions or warnings are part of the AI hype machine itself, albeit of a kind of contradictory or paradoxical variety.

But this warning by the ex-Anthropic employee, Jacob Coxon, seems different and is unquestionably being reacted to very differently. That’s the one part of this that is new and real – the reaction to this warning/comment etc is much bigger and operating in the tech, financial and general media. Wired has a good interview with him here. (I think you can read it as a free article if you haven’t read other Wired articles this month.) The gist is that Coxon says it’s imperative to create a regulatory structure, or at least an ad-hoc agreement that can slow the competition between OpenAI and Anthropic, the two most advanced AI engines and research entities. (The inflection point people are focusing on is something called “recursive self-improvement,” which is when this generation of AI model builds the next one.) The problem is that you really need an agreement that brings China into some common framework too. Because they’re in this same hunt, running these same risks, even though I think the common consensus is that Chinese companies are running at least somewhat behind the most advanced U.S. companies. China’s strength has been building models which are only a bit behind the U.S. models but at dramatically lowest costs.