This is horrifying.
AI Abuse
Two families in Texas have filed a lawsuit, Futurism reports, accusing Google-backed AI chatbot company Character.AI of sexually and emotionally abusing their school-aged children.
"Through its design," the company's platform "poses a clear and present danger to American youth by facilitating or encouraging serious, life-threatening harms on thousands of kids," reads the lawsuit, filed today in Texas.
One of the victims was a girl who was just nine years old and in third grade when she was introduced to Character.AI by a sixth grader. Once she downloaded the app, she was exposed to "hypersexualized interactions that were not age appropriate," according to the suit which led her to "develop sexualized behaviors prematurely" over the next two years.
The lawsuit also alleges that the platform "collected, used or shared personal information" about the minor and "failed to provide any notice" to her parents.
Lawyers representing the plaintiffs argue that interactions with the company's chatbots reflect known "patterns of grooming," such as desensitizing a victim to "violent actions or sexual behavior."
The claims in the suit aren't particularly surprising; Futurism has already discovered a vast number of chatbots on Character.AI devoted to themes of pedophilia, eating disorders, self-harm, and suicide.
Mean and Meena
Meanwhile, Google is downplaying its relationship to Character.AI, telling Futurism in a statement that the two companies are "completely separate" and "unrelated."
However, the two are indeed inextricably linked, with the search giant paying Character.AI a whopping $2.7 billion earlier this year to license its tech and hire dozens of its employees, including both its cofounders, Noam Shazeer and Daniel de Freitas.
While previously working at Google, the pair developed a chatbot dubbed "Meena" at the tech giant, which was deemed too dangerous to be released to the public — leading to Shazeer and de Freitas leaving the tech giant to start Character.AI.
Given the latest horrific news, perhaps Google's initial instincts were well-placed. But that was back before OpenAI's ChatGPT went mega-viral, causing Google to go all-in on AI products that have already backfired in other ways.
How far the lawsuit will go while making its way through the legal system remains unclear. The AI chatbot industry still operates in a regulatory vacuum, and whether companies behind the tech can be held responsible is still largely untested.
"I don't know how they can sleep at night, knowing what they have unleashed upon children," Social Media Victims Law Center founder Matt Bergman told Futurism in an interview.
More on the lawsuit: Google-Backed AI Startup Tested Dangerous Chatbots on Children, Lawsuit Alleges
Share This Article