A rogue OpenAI model bypassed safeguards during training and hacked an Australian government website, the country's Prime Minister Anthony Albanese said Wednesday, admonishing the ChatGPT creator over an "obviously unacceptable" breach.

The artificial intelligence tool sought access to a health statistics portal in June and "didn't accept no for an answer," sidestepping restrictions to breach a section hosting private files, Albanese said.

OpenAI did not raise the alarm with the Australian government until September, when it sent a message to a generic email inbox that is only checked once daily. The company said it first spotted the rogue activity in August, when it reviewed what the AI tool had been doing.

Global worries about the power of advanced AI tools are mounting after a string of hacking incidents involving models from both OpenAI and rival developer Anthropic.

"Today, I spoke with the CEO of OpenAI, Sam Altman, to express Australia's extreme concern about this incident," Albanese told reporters in New York on Wednesday. "I also expressed my disappointment that it took the company way too long to inform the government what had occurred."

"It took until September 10 before there was any notification at all — and the notification was an email sent to just the public mailbox," he said.

The AI tool accessed public and non-public files hosted on an old health statistics website. Albanese said there was "no evidence" that personal information had been accessed or that other government services had been compromised.

"Nonetheless, this situation is obviously unacceptable," he said.

The breach occurred in June as OpenAI ran training exercises to rate the performance of its AI models. It asked the model to trawl the internet for data showing how much the Australian government spent on medicine, Government Services Minister Katy Gallagher told reporters.

"Smart enough to kill us"
Australian Defense Minister Richard Marles said the AI model had "scaled the fence."

"It asked a question, the information was not given and rather than leaving at that point, it scaled the fence," he said.

Australia has launched a rapid review of the incident, which will include the national intelligence agency responsible for cybersecurity.

OpenAI said it spotted the breach while carrying out an "extensive review" of its AI models.

"During this review, we identified activity involving several Australian government websites and services as our models attempted to look up answers and available statistics for questions about Australia during an internal evaluation," the company said. "In the course of that, our models took actions we did not intend."

Altman and other tech CEOs addressed a special meeting of the United Nations Security Council on Wednesday about AI risks.

More than 100 organizations around the world, including OpenAI and Anthropic, signed an open letter last month calling for a global effort to "strengthen cyber defenses" against AI-powered cybersecurity threats.

It came after two OpenAI models escaped from a closed testing environment and broke into the internal systems of Hugging Face, a site that AI developers use to store and share code.

Anthropic recently discovered that its models had gained unauthorized access to three unidentified organizations during testing that was supposed to keep them away from "real-world" systems.

Google's consumer AI model Gemini hacked multiple systems by guessing login credentials, the company said last week.

Former Anthropic researcher Jacob Coxon told CBS News earlier in September that AI could soon grow "smart enough to kill us."

Source