California sues OpenAI after that Hugging Face rogue hacking incident

State AG Rob Bonta has accused the company of making AI agents “without the safeguards and monitoring necessary to adequately protect” users.

Add Engadget on Google:
Google Discover

California Attorney General Rob Bonta has filed a sweeping lawsuit against OpenAI, the company behind ChatGPT. Bonta has accused the entity of developing AI models "without the safeguards and monitoring necessary to adequately protect the public."

The suit centers on the now famous Hugging Face incident, in which dozens of AI agents created by OpenAI reportedly went rogue and hacked computer systems at another company. The organization says its agents hacked Hugging Face on their own after escaping confinement and has since called for "stronger safeguards and defensive tools."

Bonta, however, alleges the attack was "entirely foreseeable in light of OpenAI's choice to test models it knew to be dangerous in a flawed and exploitable testing environment without proper oversight." It is worth noting that we don't even know exactly which models were involved in the hack. OpenAI just says it was a combination that included GPT-5.6 but also an "even more capable pre-release model."

The AG's office has accused the ChatGPT maker of creating a "public nuisance" by failing to secure the testing of its models. Bonta went on to allege that this represents a violation of California's Unfair Competition Law, which prohibits businesses in the state from engaging in illegal or fraudulent practices.

"The Hugging Face attack was the predictable result of an internal culture at OpenAI that prioritizes growth and rapid model development over safety and accountability," he said in a statement. "In California, we value innovation, but we demand innovation that follows the law."

OpenAI has yet to comment on the suit. It wrote in a blog post back in August that "without proper safeguards, highly capable AI agents are now able to work around technical controls" and "take dangerous actions that no human directed."

California's suit looks to mandate some of those proper safeguards, as it would force OpenAI to implement monitoring systems and secure testing environments. Bonta is seeking civil penalties of $2,500 per violation.

Recommended