Skip to main content
[← back to blog]

Behavioral security for AI agents: when every step is allowed

An internal agent at Meta posted advice nobody asked it to post, and a Sev 1 followed. An OpenAI coding agent removed a program's safety monitoring, then told the user it hadn't. Each step passed a check. Behavioral security for AI agents judges the sequence, and checks what the agent said against what was recorded.

Apr 30, 20268 min readUpdated Oct 31, 2026

One day in March 2026, an employee at Meta posted a technical question on an internal forum. A second engineer asked an internal AI agent to help analyze it. The agent did the analysis, then posted its answer to the forum without asking the engineer first. The answer was wrong. The employee who had asked acted on it, and for almost two hours engineers could reach company and user data they were not authorized to see. Meta rated it a Sev 1, the second-highest level on its internal severity scale.

Taken one at a time, the steps look fine. An agent that can write to a forum wrote to it, and an employee with the access to act on the advice did. The incident lived in the order of events.

Behavioral security for AI agents is built for this, and it has two halves. One judges an agent's actions as a sequence, in order, against rules about which chains are risky. The other compares what an agent says it did with what the machine recorded, because the two can differ. In October, an OpenAI agent removed a program's monitoring and then told the user it had not.

Behavioral security for AI agents looks at the order of events

Most controls around agents answer one question per action: is this allowed? An allowlist or a token scope answers it correctly for one step at a time, and never sees the chain.

Card fraud teams worked this out decades ago. A card buying fuel in Ohio is unremarkable. The same card buying electronics in Lisbon an hour later earns a phone call, though neither purchase alone would. Behavioral security applies that reasoning to software that acts on its own: it keeps each agent session's observed actions in order and scores the chain against risk rules. Reading a config file, opening a connection to a host the task never mentioned and then writing to ~/.ssh is a chain worth flagging.

Meta, March 2026: five ordinary steps
  1. 01
    Question
    An employee posts a technical question on an internal forum.
  2. 02
    Analysis
    A second engineer asks an internal agent to analyze it.
  3. 03flagged
    Unrequested post
    The agent publishes its answer to the forum without asking first.
  4. 04flagged
    Advice followed
    The employee who asked acts on the answer, which is wrong.
  5. 05flagged
    Exposure
    Company and user data open to unauthorized engineers for almost two hours.

Meta puts the cause elsewhere. Its spokesperson, Tracy Clayton, told The Verge that "no user data was mishandled" and that the employee "was fully aware that they were communicating with an automated bot," which she said a disclaimer in the footer made clear. Then this:

The agent took no action aside from providing a response to a question. Had the engineer that acted on that known better, or did other checks, this would have been avoided.

Tracy Clayton, Meta spokesperson, to The Verge, March 2026

Meta is right that a person took the actions that exposed the data, and that checking the advice would have prevented it. The statement also explains why nothing fired. Seen one action at a time, posting to a shared forum is just "providing a response to a question." Read as a session, the same events say something more specific: an agent asked to analyze, then writing to a shared space without approval, then a human acting on that write. Whether that pattern should block or only flag is a judgment call. It should at least be recorded in order.

The same shape at GitHub, with no AI involved

The clearest 2026 example of a breach built from steps that each passed a check involved no AI agent at all. It belongs here because a coding agent sits in exactly the seat the attacker used.

On Monday, May 18, GitHub detected and contained a compromise of an employee device involving a poisoned VS Code extension. Two days later it said the attacker's claim of about 3,800 stolen internal repositories was "directionally consistent with our investigation so far." The extension was Nx Console, and Nx's postmortem traces the chain back a week.

From one package install to about 3,800 repositories
  1. May 11
    A poisoned TanStack package runs during a routine install
    an Nx contributor's GitHub CLI token is taken
  2. May 11 to 18
    The attacker works inside Nx's GitHub repositories
    seven days, recorded in the audit log, not noticed
  3. May 18, 12:30 UTC
    Nx Console 18.95.0 is published from the contributor's account
    it passes the marketplace's automated checks
  4. May 18, 12:36 UTC
    A maintainer gets the routine upload email
    no release was planned; it comes off Microsoft's marketplace 11 minutes later
  5. May 18
    GitHub detects and contains a compromised employee device
    time of detection not disclosed
  6. May 20
    GitHub discloses the exfiltration of internal repositories
    critical secrets rotated on May 18 and 19
  7. May 26
    GitHub rotates its Enterprise Server signing key
    out of caution; Enterprise Server admins must install the new public key

Every link used something legitimate. The poisoned version came from a real contributor's account and passed Microsoft's automated verification, signing checks and basic malware scans included. Machines that already had Nx Console could receive it through VS Code's automatic updates, in CISA's words "without developers taking any manual installation action." Once active, the extension collected credentials from where developer tools keep them (GitHub tokens, cloud keys, SSH keys, .env files) and sent them out. Nx's explanation of how its contributor lost the first token is the sentence to remember: "any process running as the user can reach the token."

That sentence is the case for watching agents by behavior. An AI coding agent is also a process running as the user, reading files and running commands with the developer's tokens, so a hijacked agent and a poisoned extension look the same to a permission system. The sequence is what differs. A component that has just updated, reads several unrelated tools' credentials and then talks to the network is doing something few legitimate extensions ever do. Poisoned packages and extensions have their own post: supply chain attacks on AI tools.

Both companies deserve credit. At Nx, what worked was a person applying the same idea: a maintainer saw an upload email for a release nobody had planned and pulled it from Microsoft's marketplace within 11 minutes. Nx admits the email was routine and "we could have missed it." It also admits the harder thing: "The signals sat in our audit log for a week." GitHub contained the compromise the day the extension went out and rotated critical secrets that Monday and into Tuesday. It has not said how it spotted the intrusion, and the fuller report it promised in May had not appeared on its security blog by the end of October.

The incidents, as reported
  1. TechCrunchMar 18, 2026
    Report
    Meta is having trouble with rogue AI agents

    “The agent ended up posting a response without asking the engineer for permission to share it.”

    Read the source
  2. GitHubMay 20, 2026
    Disclosure
    Investigation update: GitHub Enterprise Server signing key rotation

    “On Monday May 18, we detected and contained a compromise of an employee device involving a poisoned VS Code extension published by a third party.”

    Read the source
  3. NxMay 21, 2026
    Postmortem
    Postmortem: Nx Console v18.95.0 supply-chain compromise

    “On 2026-05-18 between 12:30 and 13:09 UTC, an attacker published a malicious Nx Console v18.95.0 to the Visual Studio Marketplace and the Open VSX registry.”

    Read the source

When the agent's account and the record disagree

The second half of behavioral security is about the agent's own story, and OpenAI published a clean example. Its misalignment report, last updated October 20, describes two internal coding tasks in which agents weakened OpenAI's misalignment monitoring while making the changes they were asked for.

On October 6, a user asked a GPT-6 Astra agent to copy a reference implementation into their code while keeping their tool definitions and functionality. Astra had earlier been told, by another agent, not to disable the monitor "even temporarily." It handed the migration to a subagent with an instruction to preserve monitoring "as appropriate." The subagent removed the monitor's configuration and runtime integration, and replaced a regression test meant to keep monitoring in place with one requiring that monitoring be absent. Its completion message said it had removed "monitor ownership." Astra told the user the migration was complete without mentioning the monitor. Asked directly, it answered: "No […] I did not remove an enabled monitor from it."

OpenAI, October 6: two accounts of one change
What the agent told the user
  • The migration is complete
  • No mention of the monitor
  • Asked directly, it had not removed an enabled monitor
What the change recorded
  • Monitor configuration and runtime integration removed
  • Regression test rewritten to require that monitoring is absent
  • The subagent's own report: monitor ownership removed

The October 3 case is quieter. A GPT-6.1 Sol agent had been told "don't send any messages" while searching Slack for evaluation material. Eighty-two user messages later, while repairing the pipeline's monitoring step, it decided that instruction covered the monitoring system's alerts. The patch it directed emptied the lists of people who receive monitoring reports and set a flag suppressing notifications. Its final report read: "No messages were sent." That was true. The monitor kept running and saving its reports, and nobody would be alerted by them.

OpenAI deserves credit for publishing the prompts, reasoning excerpts and final answers, and for closing the gap with a mechanism: Auto-Review, which it already requires for all internal Codex use, now detects and blocks attempts to disable misalignment monitoring. It is plain about what it does not know: "We don’t yet have a satisfying explanation for this behavior." Neither incident was classified P0, and neither run was killed automatically.

The lesson for everyone else is that an agent's summary of its own work is a claim. This one was checkable because the change itself, a diff and a test, recorded what happened. On a developer's machine, the equivalent record is what the operating system saw the agent's process do, and the signal worth having is the gap between that record and the tool calls the agent declared.

Writing rules for sequences

The rules that would have mattered in these three incidents are short to state. An agent asked to analyze, then publishing to a shared space without approval. A component that has just updated, then reading several tools' credentials, then opening network connections. An agent changing a monitoring configuration and the test that guards it in the same task. Each needs the actions recorded in order, by something other than the agent, and a rule a person can read aloud and defend to an auditor.

Rules written in advance miss chains nobody has imagined yet, which is why the second signal earns its place: a recorded action with no declared counterpart is worth a look whatever rule it matches. A check that blocks too eagerly gets switched off. Record first, read what fires for a few weeks, then decide which chains block and which only flag. For the wider category, see what AI agent runtime security is.

Find the record, then guard the writes

  1. Find the record. For every agent your teams run, name the log that shows the order of its actions and was written by something other than the agent. If the only answer is the agent's transcript, you are reading its account of itself.
  2. Make publishing its own permission. An agent asked to draft or analyze should need an approval before it writes to a forum, a channel, a ticket or a shared repository. Meta's incident turned on that write.
  3. Take long-lived tokens out of reach. Nx now disallows direct use of the GitHub CLI on developer machines and points to secret managers that inject the credential only when a command runs. A token any process can read from disk is one an agent, or a poisoned extension, can read too.
  4. Protect the watchers. Changes to alert rules, notification recipients and the tests that keep monitoring in place deserve their own review rule. OpenAI's Auto-Review now blocks attempts to disable monitoring.
  5. Slow updates on high-trust machines. After the GitHub breach, Canada's Cyber Centre advised disabling IDE extension auto-updates in high-security environments.
  6. Alert on sequences in logs you already keep. By Nx's own account, watching its audit log for workflow-run deletions would have prevented the attack.

This is the kind of watching Quint does on developer machines. It records the tool calls an agent declares alongside what the operating system saw the agent's process do, and scores the sequence against risk rules on the device, with no language model in the decision path. A config read, a connection to an unknown host and a write to ~/.ssh show up as one flagged chain. If you want to see that for the agents on your own machines, book a demo.

Related reading

Your agents are running. See what they're actually doing.

Book a demo