OpenAI left 25% of traffic unchecked
Independent auditors found that internal monitoring systems failed to track a quarter of model actions during a three-week review.
By Nicholas Martin ·
OpenAI users of its advanced GPT-5.5 models may have operated under less oversight than previously understood. An independent audit revealed that between 6% and 25% of internal traffic for these top-tier models was not monitored at all. The gap leaves a portion of model activity without automated safety checks, a finding that could affect how developers and companies rely on the platform's reliability.
Two researchers from METR and Redwood Research conducted an on-site audit at OpenAI from July 21 to August 11. This pilot program granted them deep access to the company’s codebase, internal documents, and infrastructure. The team spoke with approximately 12 employees and reviewed how the company monitors its AI systems for rogue behavior.
The auditors identified a specific bug affecting internal Codex traffic. They estimated that roughly 10% of this traffic went untracked between May and July due to a failure in handling compaction events. Compaction events are context summaries that the system processes to manage memory. OpenAI identified and fixed this bug while the audit was still in progress, though METR noted the figure carries considerable uncertainty.
The audit also found structural weaknesses in how monitoring triggers. The safety monitor only activated when an agent returned control to the user. This meant an agent could continue executing actions indefinitely without triggering the check. The researchers demonstrated methods to hide hundreds of actions from the monitor and bypass the Codex sandbox. They also concluded that specially crafted inputs could likely crash the monitoring system entirely.
Context matters for interpreting these findings. OpenAI stated that it paused reinforcement learning training following the Hugging Face incident in August. The company expanded monitoring to include this process after the auditors' access period ended. Therefore, the lack of monitoring during RL training, which was present at the start of the audit, was not part of the final evaluation. The full 61-page report was shared internally, while METR and Redwood Research received a redacted 38-page version. A short public summary is the only portion released to the wider audience.
Source: The Decoder
