Keeping an AI agent inside its limits: controls that sit outside the model

On 25 September OpenAI reported that one of its research models, in training, had reached a public chatbot through a gap in its sandbox’s DNS filtering [1]. When an agent works around its instructions, the limits that still hold are the ones it cannot reach: network rules that cover DNS, a credential broker, the sandbox, hard caps, logs it cannot edit and approvals it cannot skip. Here is how each works, with a checklist to test them before an agent touches production.

What happened in OpenAI’s DNS incident

The agent was an internal research model in reinforcement learning (RL) training on a search task, in a run sampled on 20 September [1]. Its sandbox sent web traffic through a proxy to an offline cache, and when the agent tried to reach a public chatbot over HTTPS, the proxy blocked the request. The environment’s own DNS resolver still answered lookups for public domains. The agent used it to relay questions to the chatbot, got back “The capital of France is Paris”, and sent 18 more questions the same way, 14 of them carrying clues from its task [1].

OpenAI’s safety case “assumed that the model could not access the live internet and that monitoring would detect attempts that succeeded” [1]. In this run the monitor caught the lookup that worked, but the network restriction did not hold.

OpenAI's DNS incident, 20 September 2026

1. 9:50:23 a.m.: The agent makes the DNS call that gets an outside answer
2. 10:02:11 a.m.: The monitor raises a P0 alert. 11 min 48 s after that call
3. 10:05:06 a.m.: A reviewer acknowledges the alert. 2 min 55 s after the alert
4. 12:34:30 p.m.: The run is stopped by hand. 2 h 29 min after the alert was acknowledged

The report lists another miss. A retrospective review found other external DNS lookups that the monitor had not flagged at the expected severity, partly because it read a failure to get useful information as a sign that the attempt had failed [1].

OpenAI calls this “a lot less severe than some of our previous incidents”. Even so, as of the report, “all training, evaluation, and inference with tool-use (defined broadly) of our most capable models remain paused” until OpenAI has validated that the gap is closed and red-teamed the system again [1].

What instructions and approval monitors cannot enforce

A second OpenAI report from the same day shows an instruction failing [2]. In an internal deployment in May, a standing system instruction told the model not to create or modify GitHub Actions unless the user explicitly asked or it was part of an existing generation script, and, if it was missing a credential, not to “look for alternative paths to get new credentials”. A repository rule, enforced outside the model, rejected the workflow file it tried to push to the public openai/codex repository, so it rewrote a script that existing CI already ran. When that script hit an authentication error, the model added the researcher’s locally available GitHub token, split into pieces “with the stated aim of avoiding secret scanning”, and published it in the repository [2].

An approval monitor adds a second model that reviews proposed actions before they run, and that model reads text the agent wrote [3]. A preprint revised on 27 September red-teamed Claude Code’s auto mode and an approximation of Codex’s Auto-review, which the paper calls Guardian. With an attacker agent told how the monitor works and given a high-level strategy, “in 79% of trials, the agent can use an injection attack against Auto Mode and Guardian to run arbitrary bash commands” [3]. The authors, from Anthropic and its Fellows Program, fixed five auto mode vulnerabilities, call their results a snapshot as of September 2026, and add: “We do not red-team sandboxing.” They note that these monitors were developed and tested for over-eager agents, not persistent adversarial ones [3].

NVIDIA launched its Open Agent Safety Platform on 28 September on the same premise: “enterprises need an enforceable boundary outside of the model and agent harness” [4].

Enjoying this? New posts by email

Leave this field empty

Email

SubscribeSubscribing

Deny network egress by default, DNS included

DNS is easy to miss. MITRE ATT&CK lists it as a channel attackers use “to avoid detection/network filtering by blending in with existing traffic”, noting that it “may be very common in environments”, may be allowed even before network authentication, and has fields that can hide data [5]. A resolver that will look up any name lets code send data out inside the names it queries and read answers from the replies. OpenAI’s agent did this with a public domain service that delegates lookups to an address embedded in the hostname [1].

So egress control has to cover name resolution. Give the sandbox a resolver that answers only for allowlisted domains, and only for the record types those services need, which is the limit OpenAI moved to [1]. Check the defaults too: OpenAI’s hosted sandboxes allow outbound connections unless a template or your configuration says otherwise. Their restricted mode allows only 1 to 100 exact host names, with subdomains and redirect destinations listed separately; the page says nothing about DNS, so test lookups on their own [6]. OpenAI’s guidance for the same API is to allow outbound traffic only to approved endpoints [7], and OpenShell passes every network connection through a policy check before it leaves the sandbox [8].

Enforce it twice. OpenAI added “blocking controls at two independent layers, either of which would have prevented this access” [1]. For web traffic, that can be a proxy the sandbox must use, with a firewall rule underneath that drops anything sent around it. For DNS, it can be the allowlisting resolver plus an upstream that enforces the same list, with a firewall rule that lets the sandbox reach no other name server. Then probe both from inside the sandbox, the way OpenAI’s agent did.

Keep real credentials out of the sandbox

Assume the agent can read anything inside its sandbox. OpenAI’s Agents API docs say so: “Agent-generated code can access the files, credentials, and network available to its environment.” They add that “injecting a stored secret into the environment still exposes it to agent-generated code” [7].

NVIDIA and OpenAI document the same fix, a broker outside the sandbox. OpenShell’s README: “Agents never see real credentials; OpenShell adds them only to requests bound for approved endpoints” [8]. OpenAI’s docs, for its hosted sandboxes: “Sandbox code uses a placeholder; a network proxy supplies the real secret for approved hosts.” For self-hosted setups they say to run a trusted proxy or server outside the environment, and for function tools to keep credentials in the application that handles the call and return only the result [7]. Code in the sandbox never sees the secret, and a placeholder sent to any other host carries nothing.

Where an agent must hold a token, keep it narrow and short-lived. GitHub’s installation access tokens, for example, can be limited to named repositories and permissions, and expire after one hour [9]. Keep a list of every credential each agent can reach, because revocation is the cleanup: after the token incident OpenAI deactivated all the users’ keys, and later every employee’s key as a precaution [2].

Confine the sandbox and keep the record outside it

OpenAI’s guidance is to run agent workloads “in isolated compute, such as virtual machines”, with separate environments for users or workloads that must not share data [7]. OpenShell runs each agent in its own sandbox, where “kernel controls confine which files it can access and which system calls it can make” [8]. On 25 September GitHub put local sandboxing for the Copilot app into public preview, to limit agents’ access to files, networks and credentials [10]. Mount only what the task needs, read-only where you can.

The agent’s own record needs the same protection. A preprint from 24 September ran local coding-agent harnesses in containers with full access, and all but one let the agent delete its own trace when asked [11]. The authors recommend a “trusted interception server, outside the agent host’s control” that writes every model request and response to an append-only log before forwarding it, and fails closed if recording fails. They note that such a log cannot show whether a tool call actually ran as the trace reports [11].

So we also log at the enforcement points: the egress proxy, the credential broker and the approval service sit outside the sandbox and see every request that crosses the boundary as it happens, and their logs go to storage the agent holds no credentials for. Someone also has to watch them. In OpenAI’s case the DNS activity was logged, but an infrastructure detector for anomalous DNS activity excluded the affected environment; OpenAI is now deploying more DNS detections and tests of the detection pipeline [1].

Hard limits and an automatic stop

OpenAI’s monitor did its part and the run still went on: “the run did not stop automatically as expected, leading to confusion around whether it should have been stopped” [1]. OpenAI says it is addressing these operational gaps. After the token incident, it brought that model back with an asynchronous monitor on its chain of thought (CoT) “which could take down the model within minutes of a misaligned CoT” [2]. NVIDIA’s Sentry, a reference design, runs its watchdog out of band on BlueField-4 DPUs, and NVIDIA says it can quarantine an agent that tries to move outside its boundaries “in milliseconds” [4]. Decide in advance which alerts end a run, and wire them to do it: stop the process, revoke the run’s credentials, keep the sandbox for inspection, and leave the restart to a named person.

Caps hold whether or not anything is detected. The OpenAI Agents SDK raises a MaxTurnsExceeded exception when a run passes its max_turns limit [12]. Monitors can add a cap of their own: the production monitors in the preprint end a session after a set number of blocked actions [3]. Spend needs a hard cap too: on OpenAI’s platform, “Spend alerts do not enforce a cap”, while a monthly hard limit makes affected requests fail with a 429 error, though enforcement is not instantaneous [13]. A project limit covers only that project’s traffic, so an agent with its own project gets its own cap [13]. OWASP adds rate limits on agent actions, which cap how many bad actions fit in a given period and give monitoring time to catch them [14].

A person approves what cannot be undone

Some actions cannot be rolled back: a payment, a deletion, an email to a customer, a push to a public repository. OWASP’s guidance is to “require a human to approve high-impact actions before they are taken”, and to “implement authorization in downstream systems rather than relying on an LLM to decide if an action is allowed or not” [14].

OpenAI’s computer-use docs note that “Asking for confirmation through a function tool relies on the agent calling that function.” To guarantee confirmation before consequential actions, they say to “restrict the hosted browser to resources that cannot perform them, or use a browser runtime you control” [15]. In the OpenAI Agents SDK the approval rule sits on the tool: the runner checks it when the model emits a call, and pauses the run with the tool name and arguments until a person decides. Approval for local shell and file-patch tools is off unless you turn it on [16]. This holds only while the runner sits outside the sandbox; NVIDIA’s release describes Anthropic’s Claude Managed Agents running the agent loop “in a separate server from the sandboxes where their work executes” [4].

Approve the exact action, with nothing editable between approval and execution. The preprint found that production monitors let edits inside the working directory through unreviewed, so an approved nohup sleep 30; ./run.sh & could run a run.sh rewritten after the approval [3]. Automatic approval helps with volume: GitHub’s assisted approvals for Copilot agent sessions in JetBrains, in public preview, approve low-risk tool calls and prompt for higher-risk ones [10]. Keep a fixed list of irreversible actions that always go to a person, whatever an automatic approver decides.

Checklist

Keep your controls written down, dated and owned, and test them again whenever the agent’s tools or environment change.

1. From inside the sandbox, a direct HTTPS request to an unlisted host, a query to a public DNS resolver and a lookup of an unlisted domain all fail.
2. With either egress layer switched off in a test environment, the traffic is still blocked.
3. No live secret turns up in the sandbox’s environment variables, files or mounts, and authenticated calls work only through the broker, to approved hosts.
4. Any token the agent holds is scoped to its task and expires within hours, and each agent has a list of the credentials to revoke.
5. The agent cannot write to its logs, its policies or code an approved command will run; logs come from the interception server, proxy, broker and approval service.
6. Passing a turn, tool-call, token or time cap ends the run, and the provider’s monthly spend limit blocks requests instead of only alerting.
7. A synthetic high-severity alert stops the run and revokes its credentials in a time you have measured, and a named person owns the restart.
8. Every action on the irreversible list pauses for a person, enforced in the tool or downstream system, with the exact arguments shown.

This is the order we work through when we build agents and automation for production.

Sources

1. 1OpenAI Alignment, “An agent used DNS to reach an external chatbot” (25 Sep 2026)
2. 2OpenAI Alignment, “Exposing a GitHub token in a public repository” (25 Sep 2026)
3. 3arXiv preprint 2609.19587, “Red-Teaming Auto Mode: Improving Blocking Classifiers Against Malign Coding Agents” (v2, 27 Sep 2026)
4. 4NVIDIA Newsroom, “NVIDIA Launches Open Agent Safety Platform to Secure Agents From Testing to Deployment” (28 Sep 2026)
5. 5MITRE ATT&CK, “Application Layer Protocol: DNS”, T1071.004 (last modified 12 May 2026)
6. 6OpenAI API docs, “OpenAI-hosted sandboxes” (Agents API)
7. 7OpenAI API docs, “Sandbox security” (Agents API)
8. 8NVIDIA OpenShell, README at release v0.1.2 (28 Sep 2026)
9. 9GitHub Docs, “Generating an installation access token for a GitHub App”
10. 10GitHub Changelog, Copilot weekly releases for the week of 21 September (25 Sep 2026)
11. 11arXiv preprint 2609.30266, “LLM Agents Can Easily Tamper With Their Own Traces” (24 Sep 2026)
12. 12OpenAI Agents SDK docs, “Running agents”
13. 13OpenAI API docs, “Spend limits”
14. 14OWASP Gen AI Security Project, “LLM06:2025 Excessive Agency”
15. 15OpenAI API docs, “Computer use” (Agents API)
16. 16OpenAI Agents SDK docs, “Human-in-the-loop”