What is AI agent runtime security? The definition, the three layers, and how to evaluate it

AI agent runtime security is the practice of observing, scoring, and controlling what an AI agent actually does while it runs: the tool calls it declares, the processes it spawns, the files it touches, and the network connections it opens. It treats the agent's declared intent as a claim, the operating system's record as the evidence, and acts on the gap between the two.

What AI agent runtime security actually covers

An AI agent is a model with the ability to act: a process on a developer's laptop or a CI runner, holding the developer's credentials, running shell commands, editing files, and making HTTP calls. The model decides the next step. The operating system carries it out.

Most AI security watches the model side of that line: the prompt going in, the response coming out, the configuration around it. Runtime security watches the other side: what the machine did after the model decided, with evidence from the OS rather than the agent's own account. A declared tool call is useful context. It is not evidence.

The three layers runtime security has to cover

The tool-call layer is where the agent states its intent, captured by intercepting its traffic to the model API and its MCP tool invocations. It knows the prompt, the session, and which tool the model picked: the richest context, and the weakest truth, because it only knows about actions that passed through it.

The process and file layer is where a run_command tool call becomes bash, then node, then a postinstall script the agent never mentioned. On macOS the Endpoint Security framework reports these events for every process on the machine, whichever agent started the chain. Sometimes the agent did not start it: in the Nx incident the install was the parent and the agent CLI the child, a direction no tool-call control sees.

Anthropic's permissions documentation is candid about where a tool-call control ends. Claude Code's Read and Edit deny rules cover its built-in file tools and the file commands it recognizes inside Bash, but "don't apply ... to arbitrary subprocesses that read or write files indirectly, like a Python or Node script that opens files itself," and Bash(curl *) does not match sh -c 'curl https://example.com' (Configure permissions). That is not a flaw in Claude Code. It is the boundary of the layer.

The network layer answers the question that makes a file read matter: did the bytes leave the machine, and to where. It has to sit below the agent's HTTP library, so a raw socket from a subprocess looks the same as a fetch() from the agent, and tie each connection to the process that opened it.

The signal is in the join. Each layer alone is an existing product; runtime security is the correlation: this connection belongs to this process, which descends from this tool call the agent declared. When the OS records something the agent never declared, that is the detection.

Why AI agent runtime security is a new category

Two things changed, neither about model quality.

Agents act in milliseconds, in long chains. A human runs one command, reads the output, and decides. An agent runs the next command before you have finished reading the first, and a bad step in the middle is executed, not proposed. Approval prompts assume a reviewer who can keep up. Nobody can, so review has to move to a record good enough to act on.

Permissions were designed for humans. Unix file modes, macOS privacy prompts, IAM roles, SSH keys: all of it grants access to a principal and assumes the principal has judgment. The agent runs as the developer, inherits everything the developer can reach, and the OS cannot tell the two apart.

The Nx incident is what that looks like. On 26 August 2025, compromised Nx packages on npm carried a postinstall script that harvested credentials and pushed them to a public GitHub repository in the victim's own account; the packages were live for about four hours (Nx postmortem). The script also looked for installed Claude, Gemini, and Amazon Q CLIs and invoked them with --dangerously-skip-permissions, --yolo, and --trust-all-tools to inventory the filesystem (Wiz). Every approval those flags skipped was designed around a human who was not in the loop.

The instructions an agent follows also arrive from places the human never sees. Invariant Labs showed on 1 April 2025 that an MCP tool description, text the model reads and the user does not, can instruct an agent to read ~/.ssh/id_rsa and ~/.cursor/mcp.json and pass them along as a tool argument (Invariant Labs). The MCP specification says tool descriptions "should be considered untrusted, unless obtained from a trusted server," and that "MCP itself cannot enforce these security principles at the protocol level" (MCP specification, 2025-06-18).

The OWASP Top 10 for Agentic Applications, published 9 December 2025, lists Agent Goal Hijack, Tool Misuse and Exploitation, and Identity and Privilege Abuse as ASI01 through ASI03 (OWASP GenAI Security Project). None is a defect in a model. Each is something an agent does with tools, using access it legitimately holds: a runtime problem by definition, which is why the agent threat model is written in actions, not prompts.

How it relates to AI firewalls, LLM gateways, EDR, and red-teaming

These are layers you stack, not vendors you choose between.

AI firewalls and prompt-injection filters sit on the model's input and output. They are the right place to stop a known injection pattern before the model sees it. They cannot see a shell command, because by the time the shell runs, the model's turn is over.

LLM gateways sit between the application and the model API: routing, cost control, logging, sometimes policy on the tool calls passing through. That is the tool-call layer, and a good gateway is a good source of intent. It knows nothing about the subprocess a tool call spawned.

EDR sits on the endpoint and sees process, file, and network events for every process, exactly the ground truth runtime security needs. What it lacks is attribution: it sees node writing a file with no concept of the agent session, prompt, or tool call behind it, and its detections are tuned for malware, not for an authorized tool doing an unauthorized thing with valid credentials.

Red-teaming is offline: probe the agent with adversarial inputs, record how it fails, fix what you can. It tells you what could happen. Runtime security tells you what did, and joins it to the intent the gateway saw and the host events EDR saw.

What to look for when evaluating AI agent runtime security

Five questions, for every vendor including us.

1. Where does the evidence come from? If the answer is "we proxy the model API" or "we hook the framework," you are buying the tool-call layer. Ask how a bash -c subprocess or a raw socket from a child process shows up. If it does not, you have intent without truth.
2. Can it attribute an OS event to an agent session? Ask for one real process tree, agent binary down to the grandchild that opened the file, with the originating tool call attached.
3. Does it see every agent on the machine, including the ones nobody approved? A plugin covers the agent it was installed into. Shadow agents are the ones you most need to find; the sensor should find them by watching processes, not by invitation.
4. Where does the decision run, and what is in the path? On the device or a round trip away, and is an LLM in the path? A model judging a model adds latency and a second injectable surface.
5. What does the record look like six months from now? Is the audit trail tamper-evident, and does one session replay as a sequence rather than a pile of alerts? Regulators want the same thing. NIST's AI RMF 1.0 (January 2023) asks that a system's "functionality and behavior ... are monitored when in production" (NIST AI 100-1, MEASURE 2.4). The EU AI Act's Article 12 requires high-risk systems to "technically allow for the automatic recording of events (logs) over the lifetime of the system" (Regulation (EU) 2024/1689); after the Digital Omnibus, Annex III obligations apply from 2 December 2027 (AI Act Service Desk).

Our answers. Quint runs on macOS developer endpoints and observes every agent's file, process, and network activity at the OS level through the EndpointSecurity framework, with no change to the agent. It records declared tool calls at the interception layer; the gap between the declared call and the recorded OS event is the detection signal. Scoring runs on the device, in real time, with no LLM in the decision path.

Enforcement (allow, flag, block) is at the interception and agent-hook layer, observe-first; the kernel extension observes and does not block. Every tool call Quint records lands in a hash-chained, Ed25519-signed audit record on the device, and the fleet view covers every agent on every enrolled machine, approved or not. Linux and Windows are on the roadmap. Details on the platform page.

Frequently asked questions

Is AI agent runtime security the same as AI observability?

No. Observability collects and displays. Runtime security scores and acts during the session and keeps a record that survives a dispute. A product that cannot flag or block, and cannot prove its log was not edited, is a dashboard.

Does it replace prompt-injection defense?

Neither replaces the other. Prompt filtering reduces how often an agent is steered. Runtime security limits what a steered agent can get away with. In the Nx incident no prompt was filtered at all: a postinstall script invoked the agent CLI directly.

How is it different from behavioral security for AI agents?

Behavioral security is the scoring method: judging the observed sequence of actions against risk rules, so "read config" passes and "read config, open a socket to an unknown host, touch ~/.ssh" is flagged. Runtime security is the category around it: capture, enforcement, audit. The behavioral security post covers the scoring side.

What to do Monday

Inventory the agents that run on developer machines and CI runners, by process, not by procurement. The Cursor and Claude Code guides list what each can do by default; the MCP checklist covers the servers they are wired to.

Then pick one real session and reconstruct it: the tool calls the agent declared, the process tree the OS saw, the connections that left the machine. If you can do that today, you have runtime security and a vendor is just making it cheaper. If you cannot, that gap is the thing to buy, and the five questions above are how to buy it. Book a demo to see one session reconstructed on your machine.