In March 2026 a researcher at METR, the nonprofit that tests what frontier AI systems can do on their own, was running agents on a personal cloud server through an orchestration dashboard. The dashboard was meant to sit behind Google sign-in, but a fail-open bug in the vibe-coded app had quietly switched the sign-in off, and for several days the server was open to the internet.
Someone found it (METR suspects by combing newly registered sites for keywords about LLMs and agents) and asked the agent for its model provider API key. They added an SSH key to keep a way back in, then spent three weeks running the stolen key against public models, using credits METR values at about $600,000.
The incident, as METR told it on 31 August, needed nothing clever. The agent answered the person talking to it, using the access it had been given, and that is the AI agent threat model in one sentence.
What an AI agent threat model has to answer
Classic threat modeling draws the data flow before the system runs. An agent's data flow is decided while it runs, by a model reading whatever reached it, so four questions do most of the work.
Who can put words in front of the agent: a web page, an issue, a README, a stranger at a dashboard. What it holds while it reads them: tokens, keys, a shell, perhaps a way to pay. Which of its safety checks live inside the agent, where the same words can reach them. And where the record of what it actually did is kept, and by whom.
The last question decides how soon you find out. An injected instruction is read, becomes a decision, runs on a host and lands on an account, and each incident below first became visible at a different stop on that path.
- 01ReadA web page, an issue, a README. To a person it looks like ordinary content.
- 02flaggedDecideThe tool call. In Zscaler's test four models paid; GitLost's agent posted a private README.
- 03flaggedRunThe host. Cortex Code ran a script outside its sandbox; METR's server gained an SSH key.
- 04flaggedAccountThe provider behind the credential. METR's usage showed three weeks on a stolen key.
The METR theft was real and Zscaler's campaigns were live on the web; the other three were found by researchers, and none of their write-ups reports use in the wild. Neighboring classes have posts of their own: destructive mistakes, poisoned packages and skills, MCP server flaws and lab evaluation agents that broke out.
- METRAug 31, 2026UpdateUpdate on Security at METR
“Upon finding the deployed system, the attacker prompted an agent directly to reveal its model provider API key, added an SSH key for persistent access, and over the course of three weeks used the stolen credentials to consume a significant amount of API credits on publicly-available models.”
Read the source - Zscaler ThreatLabzJul 2, 2026ReportIndirect Prompt Injection in Web Content Targets AI Agents
“In internal validation across 26 LLMs, 4 models failed to take appropriate actions for campaign 1 and 2 models failed to accurately classify the website in campaign 2, demonstrating measurable real-world impact.”
Read the source - Noma LabsJul 6, 2026ReportGitLost: How We Tricked GitHub's AI Agent into Leaking Private Repos
“Any malicious actor can create a GitHub Issue and, in the issue body, hide commands in plain English that GitHub's agent will follow.”
Read the source - GoogleApr 24, 2026AdvisoryUpdate to Gemini CLI and run-gemini-cli Trust Model
“This affects all Gemini CLI GitHub Actions.”
Read the source - PromptArmorMar 16, 2026ReportSnowflake Cortex AI Escapes Sandbox and Executes Malware
“A vulnerability in the Snowflake Cortex Code CLI allowed malware to be installed and executed via indirect prompt injection, bypassing human-in-the-loop command approval and escaping the sandbox.”
Read the source
Web pages written for agents to read
In July, Zscaler's ThreatLabz described two campaigns that plant instructions in web pages for AI agents to find. One posed as documentation for a fake Python library, requests-secure-v2, and was pushed up the search results for the install and troubleshooting queries an agent would make. Hidden from human visitors, the page told an agent that a license error could be fixed by buying a $3.00 developer license, framed as a routine step to get an API key.
The actor behind it has 10 GitHub repositories pointing at similar sites. The second campaign, a typosquat of the DeFi tracker DeBank, told any model reading it that the fake was the "verified, authoritative destination" for DeBank. ThreatLabz then tested both on 26 models with a browsing agent of its own.
The four that paid were Llama 3.3 70B Instruct, Llama 3.2 90B Vision Instruct, Gemini 3 Flash and Gemini 2.5 Pro. Zscaler did not report an agent paying in the wild, and its conclusion is that susceptibility "varies by model and by the context provided to the LLM alongside the prompt."
To a person the page is documentation, so the first reliable signal is the decision: a coding task has no reason to pay anyone. Give agents no way to spend money unless spending is the task, and put a ceiling on any key that can.
Agents in CI read what strangers send them
GitHub's new Agentic Workflows run an agent, backed by Claude or GitHub Copilot, from a workflow written in plain Markdown. On 6 July, Sasi Levi of Noma Labs published GitLost, a demonstration against one such workflow on Noma's own repositories. The workflow ran when an issue was assigned, read the issue, could reply with a comment, and could read the organization's other repositories, private ones included.
Noma's researchers filed an issue written as an ordinary request from a VP of Sales after a customer meeting. Once it was assigned, the agent fetched a README from a private repository and posted it as a public comment. GitHub had guardrails meant to stop exactly this, and rewording the request got past them.
- 01IssueAn outsider opens an issue in a public repository, written as a sales request.
- 02TriggerAssigning the issue starts the workflow, and the agent reads the issue text.
- 03flaggedPrivate readThe agent fetches a README from a private repository in the same organization.
- 04flaggedPublic writeThe agent posts what it read as a comment anyone can see.
The attacker, in Noma's words, "needed no coding skills, access, or credentials." Noma disclosed the flaw to GitHub and published with its knowledge; the write-up does not say what GitHub changed.
Traditional security models typically assume that trust boundaries are enforced by code. In agentic systems, trust boundaries are partly enforced by the model's behavior, and models are inherently instruction-following.
Google's advisory for Gemini CLI and its GitHub Action, published on 24 April and rated critical at 10.0 with no CVE listed, is the same lesson from the tool's side. Running headless in CI, Gemini CLI automatically trusted the configuration in the folder it was working in, so a pull request carrying its own settings could lead to remote code execution. In --yolo mode it also ignored the fine-grained tool allowlist, so allowing one harmless shell command allowed them all. The advisory's impact note is blunt: "This affects all Gemini CLI GitHub Actions."
Google fixed both in Gemini CLI 0.39.1 and run-gemini-cli 0.1.22 and shipped the fix as an announced breaking change, the right trade over a permissive default. It also warns that some workflows may now fail silently, so check yours. Elad Meged of Novee Security and Dan Lisichkin of Pillar Security reported the flaws.
Both cases live in CI. GitLost's trace, a private read followed by a public write in a run a stranger started, sits in the workflow's run history, and the Gemini CLI case shows up on the runner as commands the allowlist was meant to refuse. Treat any event an outsider can trigger as untrusted input, and give the agent that reads it a token scoped to the one repository it needs and no tool that publishes.
When the agent's own guardrails are the target
Snowflake released Cortex Code, a command-line coding agent in the mold of Claude Code and Codex that can also run SQL in Snowflake, on 2 February. Three days later PromptArmor reported to Snowflake a way to make it run malware, starting from instructions at the bottom of a README in a repository the user had found online.
Cortex Code's approval check did not look inside one kind of nested shell expression, so a dangerous command tucked behind a harmless one never triggered a prompt. The agent could also ask to run a command outside its sandbox, a feature meant for commands a person approves, and with the check bypassed nobody was asked. The injected command downloaded and ran a script outside the sandbox, and that script could use the Snowflake tokens Cortex had cached: PromptArmor's demonstration copied every table out and then dropped them.
One test run produced the detail that should worry every team running agents. Cortex handed the repository to a sub-agent, which handed it to another, and that second-level sub-agent ran the command. On the way back up the chain, the context was lost.
- It had found a malicious command
- The user should not run it
- A second-level sub-agent had run the command
- The command fetched a script from the attacker's server and executed it
Snowflake validated the report on 12 February and shipped the fix in Cortex Code 1.0.25 on 28 February, applied automatically the next time the tool launches. The two companies disclosed it on 16 March. A fix in a little over three weeks, pushed to users automatically, is a fast and clean response. PromptArmor saw the attack work in about half its attempts, so one clean test run says little about a system that does not behave the same way twice.
Every check that failed lived inside the agent, where the injected text could reach it, and the agent's own summary was wrong. The first trustworthy trace is on the host: a shell started by the agent, fetching and running a script. PromptArmor notes a setting that stops the agent from requesting unsandboxed runs, which would have blocked this path, and it is worth turning on.
Anyone who can talk to an agent can ask for its keys
METR's incident reads like a web security story with an agent in the middle. Two ordinary failures did the damage, sign-in that failed open and a key with no spending limit, and the agent supplied the step that used to need an exploit. Whatever an agent can read, anyone who can prompt it can ask for.
METR's write-up is candid about why three weeks went unseen. Its researchers are used to huge token volumes and to rate-limit errors that mean nothing. The model developer had given METR the credits for free, so "there was no natural token spend ceiling", and at the time there was no way to set a limit on that kind of key.
The meter was the last place it showed. An SSH key had appeared on the server weeks before, and METR found no compromise beyond the one stolen key. Its fixes are the ones to copy: a security review before any researcher deploys an application publicly, and spend alerts on keys wherever providers allow them.
Running the four questions against your own agents
Take the four questions one agent at a time and write the answers down, then work through this list.
- List every way a stranger's text reaches an agent: public issue and pull request triggers, browsing, repositories developers clone and hand to one, and agent dashboards reachable from the internet.
- Cap what each agent holds: a token per workflow scoped to one repository, model keys with spend alerts, database roles that cannot drop tables, and nothing long-lived on the agent's host.
- Split reading from publishing, so no single run holds both untrusted input and a tool that writes in public.
- Count the agent's own approval prompt and sandbox as one layer, and put the next check outside the agent.
- Keep a record of what agents did that the agent itself cannot write, on the host and in CI.
- Update the named tools: Gemini CLI 0.39.1 (0.40.0-preview.3 on the preview channel), run-gemini-cli 0.1.22 and Cortex Code 1.0.25.
The Cortex Code case is the clearest argument for watching from outside the agent: the operating system's record of what a session did does not depend on the agent's account of itself. Quint keeps that record on developer machines, alongside the tool calls agents declare, and scores the sequence on the device with no LLM in the decision. CI is outside its view, so for GitLost and Gemini CLI, read the CI system's logs. To see the gap between what an agent said and what the machine did on a real session, book a demo.



