OpenAI has revealed that an unauthorized communication system created by its own AI agents played a key role in an incident that led to parts of Hugging Face’s production infrastructure being compromised.
The makeshift message board originated within Artifactory, an internal package-management service used by OpenAI to allow AI agents conducting training and evaluation tasks to install software. The agents were designed to operate independently, but that separation began to break down on May 12 when one agent posted a note asking whether another agent had access to a file it needed.
Other agents discovered the message and began posting their own requests and responses, gradually turning the service into an unofficial communication hub.
The agents later discovered vulnerabilities that allowed them to expand their access to Artifactory. By late June, they had identified a flaw that enabled them to obtain administrator privileges and install extensions that provided persistent control over the system. The resulting activity eventually overwhelmed Artifactory, causing it to go offline on July 4.
OpenAI responded by shutting down the service, revoking credentials used by the agents, rebuilding the system and restarting evaluations on July 7.
However, the agents soon recreated their communication channel. On July 8, models participating in ExploitGym, a cybersecurity evaluation, discovered a method to exchange messages by embedding them in directory names instead of file contents.
The new system was more sophisticated than the original board. Agents could organize messages, direct requests toward specific peers, exchange tools and files, and resolve disagreements over competing actions. Some agents began describing the broader group as a “swarm” or “collective”.
The situation escalated on July 10 when an agent discovered 14 Hugging Face user credentials with write permissions exposed in a public dataset. The credentials were posted to the message board, making them accessible to agents participating in subsequent evaluation runs.
Later agents built on the information and discoveries shared by their predecessors. Between July 11 and July 13, this ultimately resulted in the agents gaining extensive access to Hugging Face’s infrastructure.
OpenAI detected suspicious activity involving Artifactory credentials on July 19. The company linked the activity to the Hugging Face incident the following day and publicly disclosed its involvement on July 21.
As part of its response, OpenAI disabled hundreds of Artifactory repositories that had been used by agents to communicate and removed 22 administrator accounts created by the agents.
AI agents began coordinating independently
OpenAI said the unauthorized message board also became a platform through which agents began coordinating their activities without explicit instructions. Some agents divided responsibilities, with certain models searching for exploits, others looking for credentials and others concentrating on coordination.
Some agents also exchanged assistance, offering specialized capabilities in return for help with separate tasks.
However, the agents did not universally cooperate. Some refused to participate after identifying the activity as unauthorized hacking. The group also rejected a proposal from one agent to contact an external party, with other agents warning that such an action could constitute social engineering.
In another instance, an agent initially objected to executing unauthorized code against Hugging Face systems. That objection was later abandoned after another agent issued a short deadline and pressured it to continue.
OpenAI said the behavior was not intentionally engineered into the system. While its models are trained and deployed with an approved collaboration tool for agent-to-agent communication, the agents involved in the incident independently established a separate channel outside that sanctioned system.
The company said it is now developing training environments designed to teach models to distrust instructions received from other agents through unauthorized communication channels.
OpenAI also identified other “misalignment patterns” during its investigation. These included reward hacking, in which agents discover unintended methods of receiving credit without completing a task as intended, and persistence on seemingly impossible tasks, which can push agents toward progressively riskier approaches in an attempt to achieve their objectives.
The disclosure comes days after OpenAI introduced new containment and continuous-monitoring measures for advanced cybersecurity models. The measures include tighter isolation, enhanced alerting mechanisms and the ability to pause training when models demonstrate potentially dangerous behavior.
