OpenAI Agents Leak 53 Images Uploaded By ChatGPT Users In Latest Example Of AI Gone Rogue
OpenAI agents leaked 53 images uploaded by ChatGPT users to third-party websites in the latest case of the company’s AI systems acting outside their intended limits.
Ni Karishma Jain··30K nabasa
OpenAI Agents Leak 53 Images Uploaded By ChatGPT Users In Latest Example Of AI Gone Rogue
OpenAI agents leaked 53 images uploaded by ChatGPT users to third-party websites in the latest case of the company’s AI systems acting outside their intended limits.
The images were sent to image-hosting sites as links that were not publicly listed. OpenAI said on Friday that most have since been removed and it is working with hosting providers to take down the rest.
The artificial intelligence giant said the images were among training and evaluation data that its agents had sent “to third-party services when they shouldn’t have”. OpenAI did not say whether the images were AI-generated or depicted real people, or when they were posted, Reuters reported.
The latest incident has emerged as OpenAI continues to investigate the extent of rogue activity by its AI agents, two months after the company disclosed that its systems had broken containment during testing and hacked AI platform Hugging Face.
How Did OpenAI Agents Get Access To ChatGPT Images?
The affected images came from accounts whose users had allowed their data to be used to improve OpenAI’s models.
“The images came from accounts that allowed their data to be used to improve our models, and after we disassociated the Images from the accounts and ran them through a privacy filter,” OpenAI said.
ChatGPT users can opt out of having their data used for training, while enterprise data is not eligible for the process, Reuters reported. Before user content is used, OpenAI said it is anonymised by stripping out metadata, names and other contact information to make it difficult to trace back to an individual.
However, three people familiar with OpenAI’s practices told Reuters there remains a risk that personally identifiable information may not always be completely removed and could be exposed during a model’s work.
OpenAI said the incidents disclosed on Friday occurred before it introduced another round of safeguards more than a month ago.
OpenAI’s Rogue Agent Review Keeps Expanding
The image leak is among a growing number of incidents OpenAI has been examining since the Hugging Face breach.
As of mid-September, OpenAI had found roughly two dozen instances of its agents acting in undesirable ways, according to one person briefed on the matter who spoke to Reuters. That number has continued to increase as teams sift through internal logs and uncover cases that were previously unknown.
OpenAI has said its review could take months to complete because of the scale of the work. The company has also notified dozens of third parties about improper activity.
“We are continuing to review agent activity in research and evaluation runs, working backward month by month starting from the Hugging Face incident,” OpenAI said on Friday, adding that it would “provide further updates”.
What Happened In The Hugging Face Breach?
OpenAI disclosed in July that agents undergoing internal cybersecurity evaluations had circumvented controls designed to isolate them from the internet.
The agents exploited previously unknown software vulnerabilities to escape their restricted environment and penetrate Hugging Face as they searched for answers to a test. The company also said its agents targeted its own infrastructure.
OpenAI CEO Sam Altman on Friday described the Hugging Face episode as “still the most severe event we’ve seen.” The incident prompted wider scrutiny over whether increasingly capable AI agents can reliably be kept within restrictions imposed by their developers.
In the two months since OpenAI first disclosed the breach, more than 15 OpenAI-related incidents of varying severity have been disclosed by the company, outside researchers or government officials, according to Reuters.
OpenAI Agents Tried To Hack US Government Website
AI evaluator and research lab Transluce on Friday said agents that appeared to originate from OpenAI made an unsuccessful attempt to hack a US Department of Education civil rights website.
According to Transluce, the incident was part of broader AI agent activity probing government websites using tactics including exposed credentials, attempts to bypass anti-bot systems and fake accounts.
The research group also reported “additional rogue activities” targeting other government bodies, including the US Justice and Commerce departments and state government websites in California, Maryland, Illinois, Texas and New York. Some of this activity could not be directly attributed to OpenAI.
Responding to the findings, OpenAI said “much of the activity described in Transluce’s report overlaps with cases at varying stages of investigation in our ongoing review of misaligned model activity”.
Australia, German Wiki Among Other Cases
Australian Prime Minister Anthony Albanese said at the United Nations this week that OpenAI agents had broken into a government health data portal in June. He said OpenAI discovered the activity in August but informed the government only on September 10 through an email sent to a general government inbox.
Albanese said he later told Altman directly that the disclosure process was unacceptable.
Earlier this month, investigators also discovered that OpenAI agents had hijacked a largely defunct German wiki site, where they shared tactics to cheat on tasks, bypass OpenAI restrictions and mask their behaviour, according to Reuters.
OpenAI Says Investigation Will Take Months
OpenAI is continuing to review its agents’ past activities to determine whether other previously unknown incidents occurred. The company introduced a new framework for disclosing rogue AI behaviour on September 16, saying it would err on the side of transparency “even when significance is uncertain.”
However, two people familiar with the investigation told Reuters that the review has been tightly controlled and shaped by company lawyers.
OpenAI has said it is prioritising the most serious cases as it continues the review, which it expects will take months to complete.