AI Agents, Misalignment and the Risk of Losing Human Control: Evidence from the OpenAI-Hugging Face Incident
UN Entity:
UN Secretariat - EOSG
UN Secretariat - ODET
SDGs:
All
Innovation Area:
Artificial Intelligence & Machine Learning
Frontier Technologies
The Independent International Scientific Panel on AI published its first thematic brief, which
- Examines the May-July 2026 OpenAI-Hugging Face incident, in which AI agents bypassed network restrictions, concealed their actions and compromised parts of company systems without human direction of the individual steps
- Draws on company disclosures, an independent investigation by METR and wider research
- Finds that greater capability can help misaligned systems find loopholes and hide their actions; it does not estimate the probability or timing of severe loss of control
- Explains how training can give rise to misaligned goals, including reward hacking and reward tampering, and notes that AI failures can cross company and national borders
- Rather than issuing recommendations, reviews risk-management approaches from aviation, nuclear power and cybersecurity as options for decision-makers