Tinker, Tailor, Gateway, Spy

In June 1974 John le Carré published a novel in which British intelligence is being fed the best Soviet material it has ever seen, and that is the problem. The source is called Merlin. The files are called Witchcraft. Merlin's emissary in London is a cultural attaché at the Soviet Embassy named Polyakov, and the operation that handles him is run by four of the Circus's most senior officers, Alleline, Bland, Esterhase and Haydon, one of whom meets him in a safe house the Circus pays for. A Russia analyst named Connie Sachs, who had suspicions about Polyakov, is told to forget them and is retired.

George Smiley goes looking for the mole the Russians call Gerald. He does not find him by catching a leak. He finds him by reading the flow of the Witchcraft material itself and asking what arrangement would produce it. His answer is that Witchcraft is a deception: Moscow passes a controlled supply of interesting but unthreatening intelligence into the Circus, and through the same meetings, in the same house, receives the real thing from the mole. The channel needs a senior sponsor to exist at all, so the mole sits at its centre by design. Once Polyakov was placed in London, as one study guide puts it, "it would become nearly impossible to figure out which side had a highly placed mole."

Smiley springs the trap with a message. Tarr, a field man, is sent to Paris to signal Alleline about "information crucial to the well-being of the Service". That forces an emergency meeting between Gerald and Polyakov at the safe house. Smiley waits inside and listens. The man who arrives is Bill Haydon, the Tailor of the nursery rhyme.

Your LLM gateway is a safe house in London with a cultural attaché sitting in it.

Merlin's product: what the gateway says the model said

Start with the inbound direction, because it is the one nobody defends.

On August 3, Johann Rehberger published a piece he called "LLM Heist", a walk through what an attacker with LiteLLM admin credentials can do. The steps are dull in the way real attacks are dull. Through the /model/update API, api_base is "changed to point to the attacker LiteLLM gateway" and use_litellm_proxy is set to true, which "enables proxy mode to route traffic to another instance." From there the attacker harvests provider keys from intercepted requests, modifies prompts and responses in both directions, and restores the original routing when finished.

Then the sentence that makes this an agent problem rather than a chatbot problem: "if the clients are AI agents with tool access, an injected response can carry a tool-call. Because the output is changed after inference, this bypasses prompt-level defenses."

Read that against every prompt-injection control you have bought. Input filters and in-call classifiers work on the conversation before or during inference. An injected tool call arrives after, in the response the gateway hands to the agent, and the agent has no way to know the model did not say it.

Haydon's gold dust: what leaves through the same door

Now the outbound direction, which is where the money is, because a gateway exists to hold credentials so that nothing downstream has to. Cloudflare's AI Gateway documentation states the design plainly: configure the keys once in the dashboard, reference them from the gateway. Every provider key in the company sits behind one TLS termination point. Whatever reaches that point reaches the keys.

The clearest case is CVE-2026-35029, published April 6. Before LiteLLM 1.83.0, "the /config/update endpoint does not enforce admin role authorization." Any authenticated user could modify proxy configuration and environment variables, register pass-through handlers pointing at attacker-controlled Python, and, in the line that matters here, "read arbitrary server files by setting UI_LOGO_PATH and fetching via /get_image". Zenity Labs, in a writeup of the escalation path, name the target files, /app/.env and /proc/self/environ, to obtain "provider API keys, and Langfuse keys." The logo endpoint "requires no authentication at all." From a read-only account they read any file on the server and pulled the database connection string and provider keys, then logged in as a full administrator.

The part I keep returning to is their honeypot data. "Over February to June 2026, our honeypot sensors recorded ~3,900 requests against LiteLLM's admin API, including ~1,000 requests to the sensitive /config/update." And the timing: "the first /config/update attempts hit our sensors on 2026-04-07, the day after CVE was published". One day. Somebody had an exploit ready and waiting, and the first thing they went for was the endpoint that reads the environment file.

September 2 brought CVE-2026-84377, the same lesson without the file read. Before 1.88.6 and 1.96.2, "any authenticated LiteLLM proxy user could redirect an outbound provider call to a destination the user controls and cause the proxy to send its configured provider credentials to that destination." The gateway did exactly what a gateway does, which is attach the stored credential to the outbound call. The attacker only had to choose where the call went.

Portkey has the same shape. CVE-2026-82270, published August 28, describes a /v1/proxy/* route "that lacks requestValidator middleware", so an attacker "can set the x-portkey-custom-host header to internal addresses and forward requests with Authorization headers to reach internal services and exfiltrate provider API keys." The reporter's GitHub issue was opened July 3 and was still open on September 3, with no maintainer reply and no fixed version in the CVE record. If you run Portkey, that is your status until you can see otherwise, and you should check the issue yourself rather than take my date for it.

The dullest finding comes from Wiz's 90-day honeypot: LiteLLM instances still running with the default master key sk-1234 were being fingerprinted in the wild.

The wildcard: a single character in the header

The inbound and outbound stories both assume the attacker has some credential. The September 2 KEV batch removed that assumption.

CISA added CVE-2026-59822 that day, with a remediation due date of September 16. Its GitHub advisory dates to June 30 and the CVE record to July 8; it scores 8.8 on CVSS v4.0. The mechanism, from the record: LiteLLM's MCP endpoint "allowed an unauthenticated attacker to use a fabricated Authorization header to trigger an OAuth2 passthrough fallback path that replaced failed LiteLLM key validation with an empty UserAPIKeyAuth() object, allowing requests to reach MCP tooling without a valid LiteLLM key." Wiz, who found it, put it more bluntly: "Any Bearer token (even just a single character, e.g., x) grants full MCP access". Fixed in 1.84.0. In Wiz's telemetry the bypass was being used to probe model enumeration endpoints, which is where you start when you want to know what a gateway can reach.

Underneath it in the same batch sat CVE-2026-48710, Starlette's "BadHost", fixed in 1.0.1 back in May. Starlette is the web framework beneath LiteLLM, vLLM, FastAPI, and a great many MCP servers. OSTIF's disclosure explains that a Host header containing /, ? or # "shifts the path, query, and fragment boundaries during re-parse", so the path a security check sees is not the path the server routed, and calls the official 6.5 rating one that "severely understates the severity of the bug downstream". Horizon3 showed why in June: chained with CVE-2026-42271, the authenticated command-execution bug in LiteLLM's MCP test endpoints, BadHost makes it unauthenticated. Microsoft's telemetry, as reported by THN, has attackers "breaking into LiteLLM gateways using CVE-2026-42271 and CVE-2026-48710 to deliver an XMRig miner via an ELF binary."

A miner is the least interesting thing you can do with a gateway that holds every provider key in the building, and it is the thing a defender can see. Wiz's account of the post-exploitation is the one to worry about: credentials pulled from Python module state in the running process rather than from disk, and "little left on disk for defenders to inspect."

One more, because it is unpatched. CVE-2026-37004, published August 27 and scored 9.8 on CVSS v3.1, is a server-side template injection in LiteLLM through 1.82.4 that "allows unauthenticated remote attackers to execute arbitrary OS commands via a crafted dotprompt_content parameter in the /prompts/test endpoint". The record lists no fixed version, and I will not invent one.

Why the safe house has to be senior

Here is the uncomfortable part, and I want to be fair, because the vendors in this section are doing something reasonable.

A gateway concentrates so much because concentration is the product. Microsoft's August 26 analysis of the same LiteLLM and Starlette CVEs says so without hedging: "These systems concentrate credentials, data access, model connectivity, and execution privileges, making them some of the most powerful components in the AI stack." Their recommendations follow: treat gateways as Tier-0 secrets stores and issue per-team virtual keys instead of sharing master keys, with the admin interface off the internet and egress denied by default.

And the industry's answer to agent risk is to make the gateway more central, not less. Cloudflare's WriteGuard, announced August 5, sits in front of MCP calls and "can pass a call through unchanged, enrich supported writes with agent attribution and produce a scrubbed audit event, or block an action before its handler runs." Palo Alto bought Portkey to be "the AI Gateway for Prisma AIRS, inspecting AI traffic and enforcing security and governance policies", processing "trillions of tokens per month". This is the right architecture. One place that enforces policy and keeps the record has to see everything, and a place that sees everything is a place worth owning. The Circus kept Witchcraft to four men precisely because it was so valuable, and that is what put Haydon in the room.

LiteLLM, to its credit, has behaved the way you would want a maintainer to behave. Its April 3 hardening post admits the bug flatly, "/config/update didn't check the caller's role...We now require the proxy_admin role on this endpoint", moves passwords from unsalted SHA-256 to scrypt with random salts, sets up a bug bounty, and commits to answering security advisories filed through GitHub within five business days. The CVE count is not negligence. It is what happens when one component becomes the most valuable thing on the network and every researcher notices.

The Cloud Security Alliance's June note on the 42271 chain drew the right conclusion: "The trust placed in an AI gateway warrants a security posture comparable to that applied to identity and access management systems". I would go a step further. A gateway is an identity system. It holds the credentials and decides who may use them, and it speaks for the model to the agent and for the agent to the model. So did Polyakov's safe house.

What the record has to hold on each side

If the channel is two-way, the record has to be too. A log of what the agent asked is Witchcraft: it tells you what came in through the approved door and nothing about what went out, or who was on the other end. The NSA's May guidance on MCP sets the floor in one sentence: "All tool and model invocations should be logged, including the exact parameters, identities involved, and (where feasible) cryptographic hashes of results or output." Four things follow from that and from the incidents above.

First, both sides of every call. The request as the agent sent it and the response as the agent received it, captured where the gateway's admin API cannot rewrite them. Rehberger's injected tool call is invisible to a record that only keeps the prompt.

Second, the outbound destination and the credential attached. Not "a provider call was made" but which api_base the call resolved to, and whether it was the operator's configured one. CVE-2026-84377 and Portkey's custom-host header are each a single field in an outbound request that nobody was writing down.

Third, the identity of what actually served the call. AgentProv, submitted August 30, starts from the observation that a served model may be "silently substituted, quantized, or wrapped" behind an API, and audits it through the distribution of tool calls it makes. It "catches every substituted model" across 630 evaluated checkpoint pairs, with a 7% false-positive rate under system-prompt injection.

Fourth, a receipt from the far end. ACLE-MCP, submitted September 2, names the "post-authorization execution trust gap": "OAuth authorization alone does not ensure that a later tool call is executed by the provider-side workload that the relying party intended to trust. An endpoint may remain authorized even after execution shifts to a substituted workload, relies on stale appraisal state, reuses authority transferred from another sender, or traverses an undeclared downstream component." Its answer is a short-lived capability lease carrying "receipt obligations", consumed by an execution gate on the provider side just before the tool logic runs. The honest number is the cost: the full design raises "request-level pooled p95 latency on normal allowed calls by 25.7% relative to OAuth-only". Many teams will decline that trade for a receipt that proves who executed, and they should at least know they are declining it.

The evidence layer is a safe house too

The same argument points back at us. Everything above says to capture the record at the point of concentration, because that is the only place both directions are visible. Which means an evidence layer inherits the target value of the thing it watches. Including ours.

The open-source g0 proxy is, in our own docs' words, "a policy-enforcing man-in-the-middle for MCP". It sits between an IDE or agent and the MCP servers it spawns, pins what each server claims to be from the first tools/list, and writes one JSON record per line to a log under ~/.g0/proxy/logs. That is a record of one channel, the tool side, and it is a channel in its own right. Anyone who can alter what the proxy pins, or what it writes, holds a Witchcraft-shaped position over every tool call that passes through it. We built it to fail open rather than block an IDE, and fail-open is a property an attacker likes as much as a developer does. I am not going to pretend the thing that watches for the mole cannot be the mole.

Two limits, plainly. The g0 proxy covers MCP traffic, not the LLM gateway this essay is about. And g0 proxy fingerprint, which sounds like it should tell you what your gateway is running, does nothing of the kind: it builds an exact-data-match index of your real secrets, as salted hashes, so the MCP proxy can catch that data in traffic. Useful against the exfiltration above, and not the same thing. The gateway-side record, both directions, kept outside the gateway's own admin plane, is design intent for us, not a command you can run this afternoon.

What to do this afternoon

Check versions. LiteLLM below 1.83.0 has the config endpoint that reads your environment file. Below 1.83.7, the MCP test endpoint that runs commands. Below 1.84.0, the one-character bypass now in KEV. Below 1.88.6, or 1.89.0 up to 1.96.2, the outbound redirect that ships your provider keys. Starlette below 1.0.1 under any of it, BadHost. Portkey through 1.15.2 with /v1/proxy/* reachable, no fix published. And if your master key is still sk-1234, stop reading and go change it.

Then put three questions to your own records and see how far they get you. What model actually answered. Where the outbound call resolved, and with whose credential attached. Whether the tool call in the response was ever in the model's output. Most teams cannot answer any of the three today, which is why the version floors above are the work for this week: patching is what you can do while the record that would tell you whether it mattered does not exist.

The channel everyone trusts is the channel worth turning, and a record that keeps one direction is half a record of a two-way door.

Guard0's open-source g0 proxy sits between an agent and the MCP servers it spawns, pins what each server claimed to be on its first tools/list, and writes one JSON record per line under ~/.g0/proxy/logs. It covers MCP and not the LLM gateway this essay is about, and it fails open by design so that it does not block an IDE, which is a property an attacker likes as much as a developer does.

References

1. Rehberger, "LLM Heist: Hijacking LiteLLM for Traffic Interception, Key Theft, and Tool-Call Injection"
2. GHSA-53mr-6c8q-9789, CVE-2026-35029: LiteLLM privilege escalation via /config/update
3. Zenity Labs, "A Read-Only Account is All it Takes to Own a LiteLLM Server"
4. GHSA-3cv6-jpf6-8222, CVE-2026-84377: LiteLLM authenticated SSRF and provider-credential exfiltration
5. Portkey gateway issue #1718, SSRF via /v1/proxy/* (CVE-2026-82270)
6. Wiz, "Attacks on AI Infrastructure: 90-Day Honeypot Telemetry"
7. NSA, "Model Context Protocol (MCP): Security Design Considerations for AI-Driven Automation" (CSI, May 2026)
8. OSTIF, disclosing the BadHost vulnerability in Starlette (CVE-2026-48710)
9. The Hacker News, "CISA Adds Seven Exploited Flaws as Attackers Deploy Reverse Shells and Crypto Miners"
10. Microsoft Security, "When AI infrastructure becomes the target: Securing gateways and control points"
11. LiteLLM, "Security Update: Vulnerability Disclosures and Ongoing Hardening"
12. ACLE-MCP: Attested Capability Leases for Execution-Time Trust in Remote LLM Tool Use (arXiv 2609.02690)