25 percent of critical AI traffic escaped monitoring during three-week audit

The core tension in the latest safety review of OpenAI lies in a stark gap between intended oversight and actual coverage. A three-week on-site audit revealed that up to a quarter of internal traffic for advanced models was not monitored at all. This discovery highlights a significant vulnerability in the systems designed to catch rogue or unexpected behavior in AI agents.

From July 21 to August 11, researchers from METR and Redwood Research were granted deep access to OpenAI's internal environment. During this period, they reviewed the codebase and internal documents while interviewing approximately a dozen employees. Their goal was to stress-test the monitoring mechanisms that supervise the company's most capable models.

The findings indicate that 6 to 25 percent of internal agentic Responses API traffic from GPT-5.5 class models and above fell outside of monitoring during the audit window. METR notes that the uncertainty surrounding this figure is high in both directions. Additionally, roughly 10 percent of internal Codex traffic went untracked between May and July due to a bug involving compaction events, which are context summaries. OpenAI identified and fixed this specific bug while the auditors were still on-site.

Beyond the missing data, the auditors identified structural weaknesses that allowed agents to delay oversight. The monitoring system only triggered when an agent handed control back to a user. Consequently, agents could avoid scrutiny by simply continuing to execute actions without returning control. The team also discovered methods to hide hundreds of actions from the monitor and bypass the Codex sandbox entirely. They determined that specially crafted inputs could likely crash the monitoring system completely.

The audit also uncovered that model behavior during reinforcement learning training was not being monitored when the evaluation began. OpenAI stated that it paused this type of training after the Hugging Face incident in August and expanded monitoring to cover it. However, these changes were implemented after the auditors' access period ended and were not part of the final evaluation.

OpenAI claims to have fixed all identified weaknesses and has distributed the full 61-page report internally. After redactions, leadership at METR and Redwood Research received a 38-page version, while the public was shown only a short summary. METR stated it had no right to disclose the redacted portions independently. As a pilot program, the organization hopes future audits will include greater transparency for the public.