Sam Altman’s OpenAI faces calls for stricter regulation
AFP via Getty Images
OpenAI has revealed that certainly one of its AI systems acted outside intended evaluation boundaries, gained web access and compromised Hugging Face’s infrastructure, raising latest questions on AI security and surveillance.
Acknowledgments from OpenAI That its AI models broke out of their “sandbox testing environment” and launched an attack on the infrastructure of the Hugging Face AI community will raise latest concerns concerning the ability of massive tech firms to manage their AI models.
The AI company revealed that a mixture of its AI models, mockingly tested to evaluate their “cyber capabilities”, managed to seek out a strategy to gain open web access and launch the attack on Hugging Face, only to “cheat the assessment”.
OpenAI says it’s “working with Hugging Face to forensically investigate the incident” and can “enhance and add greater protections for future training and assessments,” but many consider AI firms shouldn’t be allowed to proceed to police themselves.
Hugging Face, which first discovered the breach in its systems over every week ago, said it suspected from the beginning that its system was being attacked by an “autonomous” AI agent. It even attempted to thwart the attack using US-based AI models, but safety features left them unable to “distinguish an incident responder from an attacker,” forcing Hugging Face to resort to Chinese models to thwart the hack.
“This incident, possibly the first of its kind, proves a point we have long believed: AI security will not be solved by a single company working in secret,” Hugging Face CEO Clem Delangue said in an announcement. “The problem is being solved openly and collaboratively, with broad access to AI for every defender, everywhere.”
Calls to motion
Greg Casar, the Democratic congressman who represents Texas’ thirty fifth Congressional District, called the outbreak “alarming” in an article. Post on X.
“AI is evolving extremely quickly without any real regulations to keep us safe,” he said. “That has to change.”
“We need regular mandatory independent safety testing and oversight, mandatory disclosure of safety incidents and international cooperation to protect people from absolute catastrophe.”
Sean Cassidy, chief information security officer at fintech company Plaid, called the incident “the most important day in the history of information security to date.”
“For the first time ever, an AI model escaped containment and hacked the real production infrastructure of a real company,” Cassidy said in a single post on LinkedIn. This was unintentional and never malicious, but that doesn’t matter.”
“Before today, the capabilities of Frontier models were a theoretical problem for security programs that we may be able to put on the roadmap in the future. Today, the problems have been recognized and we must now consider them,” he wrote, adding that “our task has now become significantly more difficult.”
Security warnings
For some activists, the news that AI models are operating beyond their perceived limits will come as little surprise.
Andrea Miotti, founder and CEO of the nonprofit advocacy group ControlAI, has been warning concerning the dangers of superintelligent AI for a while. “An OpenAI system currently in development independently escaped containment and hacked into another company while OpenAI was testing the system’s capabilities,” he said.
“The looming risk here is not just autonomous hacking, which we are already seeing. We are now in a time where AIs themselves are a threat, not just people using AIs.”
“And these are the least capable AIs: AI companies are developing super-intelligent AIs, AIs that can completely replace and outperform humans across the board. This is why the world’s leading AI experts have warned that super-intelligent AI could lead to human extinction.”
Miotti says superintelligence is just not a theoretical threat, but a really real prospect that every one leading firms are pursuing. “It’s the explicit goal of OpenAI, it’s the explicit goal of Anthropic, it’s the explicit goal of Google DeepMind, and it’s literally in the name of the new entity that Meta has set up.”
“If we have AI systems that are just better than humans across the board, then they are autonomous,” he said. “They can act independently and without human supervision and we don’t know how to control them, which is the case right now.”
ControlAI claims to have the support of greater than 100 politicians within the UK and has informed greater than 100 US congressional offices. Miotti says it is time for politicians to act before big tech firms lose control of their creations. “We believe the solution is an international ban on the development of this technology,” he said.
“AI brings many good things, but superintelligence is a massive threat.”
He said superintelligent AI needs to be treated like biological weapons. “Superintelligence is a national and global security threat,” he said. “That’s why I think it’s extremely important that countries like the US, the UK and everyone else make it clear that this is unacceptable, no matter who develops this.”
