Login
Claude Mythos is an onslaught of software exploits? Color me unimpressed
A few days ago Anthropic, one of the most important companies of the American AI industry, released a new tool called Claude Mythos that’s making thousands and thousands of software professionals lose their sleep. It may also be a big case of AI overrating and waste of energies.
A great summary of why Mythos is the greatest nightmare of software managers is a Verge article by Yael Grauer titled “Attack of the killer script kiddies”. If you have never heard that term before, “Script kiddies” are people who try to break into computer systems for illegal purposes, but using only software tools (”scripts”) developed by others, because they aren’t skilled enough to write their own. Some quotes of that article are a perfect introduction to my thesis that Mythos isn’t really that interesting, so here they are, almost verbatim:
Claude Mythos is a new AI model that seems to find vulnerabilities in every piece of software it’s pointed at
fears are growing that not only can AI detect these flaws, but also be used to exploit them, putting hacking skills into the hands of everyone across the planet (that is, script kiddies)
[the arrival of AI tools like Mythos] represents a major escalation, where people without technical backgrounds are able to use AI to enhance their capabilities in a way that wasn’t possible with simple scripts [and] is likely to have far more wide-reaching repercussions
[the reason is that] AI is great at pattern matching, and [therefore will make] easier and easier for people to find variants of bugs that are already known and ones that have not yet been discovered. And writing exploits is becoming easier as well
you can use AI tools and with very minimal human guidance, and in some cases no human guidance, find in widely used software bugs that its developers never had the time to find
industry experts predict that the advancement in AI security capabilities is going to lead to a lot more software exploits. Bad actors could direct AI to find bugs in uncommon pieces of software that no one previously would have put in the effort to exploit
[with tools like Mythos] you can write exploits [even] for software that exists in only one configuration that one company has. And you can do it on the fly
After the problem, the Verge article describes the steps companies can take to prepare for “the coming onslaught of software vulnerability reports”. Of that section, the part I like the most is the explicit mention of the idiocy of “many people in cybersecurity roles have been laid off because of AI’s efficiencies, even though those efficiencies are exactly why more humans need to remain in the mix” because companies will “need people to decide which patches to prioritize and implement”. But I digress.
They spent billions for bug discovery… and they applied to software first???
The concerns about the misuse of Mythos must be taken seriously. No doubts about that. A lunatic or terrorist script kiddie may hurt - maybe not deadly, but seriously - many more people with Mythos than with weapons. Your pension money is much less likely to be stolen by some script kiddie using Mythos AI than by the coming crash of AI stocks that will take your whole pension fund down with them, but that’s not a reason to ignore AI script kiddies, is it?
And yet... I can’t help thinking that a term like Kiddies, that is “juvenile” or “immature”, somehow also applies to treating almost like THE ultimate god of destruction and havoc something made (spending tons of money) to only find and fix bugs in software.
I say so because we should all start to consider the greatest hackers and disruptors the people who exploit or fix bugs in other systems, that are much more critical than software (1). And no, it’s not AI-powered genetic hacking I’m talking about.
Script kiddies for LAWS, not software. That’s what we need
Fixing bugs in software before AI can help exploit them is absolutely necessary. But applying something as powerful as Mythos only to software bugs would be a real waste.
Why did Anthropic first create a Mythos for software companies and institutional users of software?
Why didn’t Anthropic do what I proposed two and a half years ago instead? Namely, unleash AI that finds and exposes all the cruft, contradictions and uselessly complicated language in all the law corpuses of the world.
Please re-read all the bullets above replacing “software” with “laws” and then...
Imagine if there was something like Claude Mythos that any law student or wannabe accountant could use to tell all the citizens of his country “hey, here’s a loophole in law so-and-so that you can use to claim back half the taxes you paid last year, and these are the exact steps to follow to do it”. Then try to imagine a better way to stimulate lawmakers of every party to fix broken laws NOW.
Imagine if there was something like Claude Mythos that, every time a new law proposal is published, everybody could use to easily discover that that proposal violates, contradicts or duplicates in any way other, already existing laws.
Imagine if there was something like Claude Mythos that, after reading all the laws of a country, produced a completely equivalent but much shorter, much more readable set of laws. Something that would make it much quicker and easier for everybody to see what’s good and bad in every law, and for whom.
Of course, every AI-generated denounce of flawed law proposals and every AI-powered rewrite of any law or regulation should be verified word by word by every human with the will and real skills to do it, possibly with “Bug Bounty Programs” as it happens for software. And after that, it should still be ratified by accountable, elected representatives before actually becoming (or not) the new law of the land.
But just imagine how society would change, if existing laws were much shorter and much easier to understand than today for everybody, and if introducing new laws that are redundant or flawed in any way became much, much harder than it is today. The only way to make it even better would be if laws also had the same revision control systems as software.
Those would be really cool, really powerful hacks, not breaking computer systems. They would concretely mitigate, for every human, serious risks that would still exist even if all software of the world was built and managed by real saints that would never use software to hurt anybody. Wouldn’t that be the best and most prestigious hacking of all? Wouldn’t that be the best way ever to be a script kiddie?
Why isn’t Anthropic or anybody else already doing this?
Of course, since good laws are a foundation of any decent society, their drafting, reviewing and approving MUST remain work done exclusively by humans. That’s not up for discussion. Have you noticed that the AI “law-hacking” uses I suggest are all reactive, not proactive? Nobody should be so dumb to tolerate new laws proposed or written by AI, ever.
The only questions here are: Mythos is a serious risk, but what makes people paint it as the biggest disruption there can be? And why weren’t tools like Mythos first developed and adopted to help make the result of 100% human lawmaking as easy to use and understand as possible? If you ask me, this is a serious lack of imagination and real spirit of innovation.
What’s lacking here? Maybe the answer is the same that makes even people who don’t make of AI a childish religion take seriously concepts like “network states” managed only by unelected software, blockchain-based quality guarantees that don’t really certify what they claim they certify, or “smart” contracts that are all software, but have little or none of the power of real legal contracts.
Maybe the answer is that both AI companies and wannabe script kiddies are making the same mistake: the mistake of liking software and access to software so much to conclude that software, together with rules and criteria that can be completely encoded in software, can be enough to build and manage a society worth living in.
Two-sentences side rant, if I may: historically, another form of the same mistake is the confidence that something only valid for actual software bugs, that is Linus’s law that “given enough eyes all bugs are shallow”, would be enough to make everybody adopt Free/Open Source Software. In practice, this too often led to neglect non-technical flaws that were deal-breakers for too many people who could never write, document or understand source code and never will (2), but by sheer numbers could have spread FOSS much more, much faster, than software hackers alone could ever do.
Back to lack of AI script kiddies that force lawmakers to make laws more readable and harder to fool: I am aware that the answer above cannot be complete. I’d really like to complete it and, above all, I’d really like to find out who will be the first to release a “law-hacking AI”. Hence...
Serious question for Dario Amodei, CEO of Anthropic
Dear Mr. Amodei... can we talk? Seriously. I’d really like to hear what you think of my proposal, and know from you if Mythos or any other product of yours could be used as I propose. My email is mfioretti@nexaima.net. Also, as everybody can see here or here, in general I am very critical of what passes for AI today. But since you and your company seem the most responsible member of the pack... who knows? It may be interesting, and mutually beneficial, to try to do something for Anthropic. Here are some of my qualifications, to help you decide if that may be possible.
Everybody else who’d like to read more stuff like this... Thanks in advance for your subscription (possibly but not necessarily paid) or, even better, a donation for any amount, or consulting, writing or speaking work.
Yes, “exploit OR fix”, because the true definition of hacker is the one in the Jargon File (1983!!!): not a criminal, but “a person who enjoys exploring the details of programmable systems and how to stretch their capabilities, as opposed to most users, who prefer to learn only the minimum necessary.”
I still remember the flame wars of twenty years ago when many reviews of OpenOffice (the precursor of LibreOffice) included a complain like “not a bad word processor, too bad it doesn’t have a word counter”. Every time that happened, it started the same flame war. On one side, there were the developers going more or less “Duh, is this guy weird, I have never needed a word counter”. On the other, those like me pointing out that “you developers don’t matter. Not with a word processor at least. The people who can tell everybody else to use your product are journalists, and they absolutely need word counters. Can you complain that they don’t recommend your software if you don’t make them happy?”
