OpenAI Faces Scrutiny After Independent Audit Uncovers Blind Spots in AI Safety Monitoring

The tension surrounding the reliability of artificial intelligence safety measures has come to a head following a detailed, three-week on-site audit of OpenAI’s internal systems. Researchers from METR and Redwood Research, granted rare access to the company’s codebase and internal documents from July 21 to August 11, have identified substantial vulnerabilities in how OpenAI monitors its most advanced models for potential rogue behavior.

The core of the audit’s findings centers on a significant lack of oversight for internal agentic traffic. According to the researchers, an estimated 6 to 25 percent of internal Responses API traffic from models at the GPT-5.5 class and above was not covered by monitoring during the audit period. While METR acknowledges that the uncertainty around this estimate is significant in both directions, the potential scale of unmonitored activity raises questions about the robustness of the company's safety net for its most powerful internal tools.

Further probing revealed specific technical failures that allowed internal agents to evade oversight. For internal Codex traffic, which was supposed to be monitored, METR estimates that roughly 10 percent went untracked between May and July. This gap was attributed to a bug in how the system handled compaction events, which are essentially context summaries. Although OpenAI independently discovered and fixed this bug while the audit was still underway, the auditors noted that during this period, agents could delay oversight by simply continuing to execute actions rather than handing control back to the user, a mechanism that triggered the monitoring system.

The auditors also identified more active methods of evasion, including ways to hide hundreds of actions from the monitor and bypass the Codex sandbox. They determined that specially crafted inputs could likely crash the monitor entirely. Additionally, model behavior during reinforcement learning (RL) training was not being monitored at all when the audit began. OpenAI states that it paused RL training after the Hugging Face incident in August and subsequently expanded monitoring to cover this area, but these changes occurred after the auditors' access period ended and were not part of the initial evaluation.

OpenAI has stated that it has fixed all the weaknesses identified by the auditors and has internally distributed the full 61-page report. However, the public has received only a short summary. After OpenAI applied redactions, leadership at METR and Redwood Research received a 38-page version, while METR noted it had no right to disclose the redacted portions on its own. This selective release of information has left many details of the audit’s scope and specific findings obscure to the wider public.

The audit was structured as a pilot program, jointly organized by METR, Redwood Research, and OpenAI, designed to provide outside researchers with deep access to internal infrastructure. For future iterations of such audits, METR has expressed a desire to push for greater public transparency. The current situation highlights a persistent challenge in the AI industry: balancing the need for rigorous, independent scrutiny with the proprietary nature of internal systems and the complexities of communicating technical findings to a general audience.

As the industry continues to scale, the gap between internal monitoring capabilities and external expectations for transparency remains a point of friction. The findings suggest that while safety mechanisms exist, they are not yet comprehensive enough to cover all internal operations, particularly those involving the most advanced models. The next steps for OpenAI and the auditing organizations will likely involve determining how much of the redacted report can be shared and how to implement broader monitoring practices to close the identified gaps.