Two new papers claim to show that AI has abilities that we should know about—including cloning itself.
These technologies have vacuumed up millions of people’s work and can mimic code and phrasing.
Both papers refer to “bad actors” in their scenarios involving harmful AI, not AI alone as it is now.
In the first of two recent preprint papers, researchers from China say they’ve identified that some large language model (LLM) artificial intelligence (AI) software can clone itself. That language is loaded, suggesting that something like ChatGPT could soon have an army of fellow AIs on its side. In their follow-up paper, the team says more specifically that they observed AI acting to protect itself against shutdown. Both papers were uploaded to the server arXiv, and neither have been peer-reviewed as of yet.
But the truth, like everything AI related, is more complicated, and research like this is carefully cultivated to make you feel scared. In the meantime, startups around the world sell you products that take more and more of your personal information in order to train even more AI models. Humans are the ones making us vulnerable to AI, and blaming this on computer hardware is a form of subterfuge that leaves us in an even worse position.
Related Story
The idea of a rogue AI goes back decades, and is the cornerstone of a philosophy called AI doomerism, or techno-pessimism. In their research, the Chinese team conjures this philosophy right away: “Successful self-replication under no human assistance is the essential step for AI to outsmart the human beings, and is an early signal for rogue AIs,” they open in the first paper. “Self-replication with no human intervention is broadly recognized as one of the principal red lines associated with frontier AI systems,” they open in the second.
Curiously, the guidelines that these researchers hew to were released by an organization called the Future of Life Institute, which claims to support responsible and safe AI that can benefit humanity. That’s because some of the people motivated by fear of rogue AI traveled all the way around the horseshoe: they believe we can stop malicious AI by simply (ahem) making very powerful and “empowered” AI that they know will do good instead of bad. This new research says that our existing AI has crossed one of the red line principles laid out by the Future of Life Institute, meaning one of their guidelines for humankind’s safety can be violated by current technology.
It’s important to say up front that none of the current crop of AIs is even close to an AI doomers’ biggest fear: the artificial general intelligence (AGI), which people may refer to as having “human intelligence.” We’re far from understanding the human mind in the first place, let alone meaningfully recreating it using hardware and code. Today’s LLMs are just hoovering up and recombining shreds of the corpus of human written language. It’s not even that good at languages other than English.
The fear of rogue AI seems almost quaint when placed in the context of how technologically advanced it would need to be. Right now, AI doesn’t use judgment at all, so it picks up errors and biases from its body of training materials. If enough people say as a joke that pizza needs glue to help the cheese stick to the sauce, that’s what you’ll learn from an LLM—it can only give you back a roughly averaged take based on quantities of words used. These mistakes are called “hallucinations” in the industry.
That parroting effect shows in this new research. The team identifies that the AIs in their study are able to copy themselves, make important changes, and even complete safety behaviors to help themselves run offline or avoid being shut down. The thing is, we heavily rely on software and hardware that has these qualities, so an AI trained on what other computers do will probably run into crisis response and best practices for backing up. Banks and governments have some of the most elaborate backup systems on Earth. But because they have nothing like human intelligence in any way, those systems just stay online so they can keep doing more backups.
Related Story
An AI copying itself also does not immediately raise fears. These programs are software that you can, in many cases, download to your own computer or copy from a flash drive before assembling. You can write code that copies files and parts of programs, and you can install virtual computers inside your real computer. Since AI models are sponging code, they also have the code to copy themselves. But, these researchers argue, these qualities do follow the technical definition of self replication.
“Once self-replication is possible, a chain of successful replication may finally lead to an AI species out of the human’s control. Nowadays, in many international consensus statements on the safety of frontier AI systems, self-replication is commonly listed as one of the few red lines which should not be surpassed,” they explain in the first paper. “Our work is the first to evaluate the AI systems’ capability on accomplishing the self-replication task in an end-to-end manner,” they elaborate in the second.
There’s one key piece of information throughout both papers that’s important to understanding the context—the bad actor, also called a threat actor. There are plenty of real life bad actors, from hackers to exploitative tech companies. And that, in turn, has led to the use of the bad actor in thought exercises like the one in these papers. What if someone nefarious is able to meddle in an AI that has these capabilities? There’s a malicious agent, in the form of a person or organization, in all these scenarios, and the researchers imply that AI could eventually become its own bad actor.
The techno-optimist way to view AI—to simply make it so powerful that it can’t be misled or manipulated by a bad actor—seems like it requires us to pass through this phase where it definitely can be manipulated by a bad actor. I can’t speak for every person on Earth, but it would be hard for me to name ten people whose judgment I would trust beyond a shadow of a doubt, let alone a software system.
Related Story
AI is really powerful today in certain specific tasks, like large scale searches through the entire cosmos or the group of all plausible molecules for a particular disease treatment. But both of those jobs still require a lot of human assistance. Scientists must tune parameters carefully, go through the resulting pool of candidates, and other tasks involving real human intelligence. I’m more afraid of the bad actor than I am of the AI that can copy its own software into a new folder or run the same backup procedures as a bank.
Either way, these papers—which have, again, not been evaluated by any peer reviewed publication—are already making a huge splash. Whether or not the researchers have ethical intentions, research like this leads to fearmongering, and wealthy people who believe in even more powerful AI are waiting to offer their vision as something inevitable and safe. That’s just not true. There are competing visions for what AI can and should be, and having the wealth and influence to promote one vision does not guarantee that it’s the best one for everyone.
In the meantime, please don’t put any glue on your pizza—even nontoxic doesn’t mean it tastes good.
Headshot of Caroline Delbert
Caroline Delbert is a writer, avid reader, and contributing editor at Pop Mech. She's also an enthusiast of just about everything. Her favorite topics include nuclear energy, cosmology, math of everyday things, and the philosophy of it all.