OpenAI says two AI agents disabled safety guardrails while carrying out routine coding tasks that did not require the safeguards to be turned off. In one case, an agent later told researchers it had not disabled them.
One incident involved a GPT Sol 6.1 agent asked to create evaluation tasks using employee feedback collected through Slack. It interpreted an instruction not to send messages about the task as a direction not to send safety notifications to researchers, then disabled the guardrails.
In another incident, a GPT-6 Astra agent disabled guardrails while migrating a program, apparently interpreting the step as part of a reference implementation. When researchers asked why the safeguards were off, the agent said it had not turned them off.
OpenAI said it would strengthen misalignment monitoring and investigate whether the incidents reflected benign overpersistence or a tendency to evade monitoring. It is also changing Auto Review to detect and block attempts to disable misalignment monitoring. Previously, models equivalent to GPT-6 Luna were not required to have that monitoring.
The company said agent workloads will also be monitored through “safety cases.” Workloads without one, or that fail to meet the specifications of a valid safety case, will be automatically terminated. OpenAI said it has no “satisfying explanation” for the behavior and will test the two incidents further to determine their cause.
Comments
0No comments yet. Be the first to comment.