The Cloudflare KV cache leak sent fragments of other customers’ prompts back to Workers AI users of the GLM-5.3 model. Between August 27 and September 9, 2026, 155 responses carried stale data from unrelated requests. Each one still arrived with a normal HTTP 200 status. Cloudflare disclosed the incident on October 26, along with fixes in SGLang, the open source inference engine behind the deployment. Any team sending sensitive context to a hosted model should take a close look at this failure.
What Is the Cloudflare KV Cache Leak?
The incident exposed cross-tenant data inside Workers AI, Cloudflare’s serverless inference platform. It touched only GLM-5.3, an open-weight model from Z.ai, and only very long prompts. When a prompt exceeded 681,280 tokens, the model could build its answer from memory left behind by another request. As a result, a response might include pieces of another customer’s prompt or partial output.
Security researcher Gadi Evron of Knostic reported the problem through Cloudflare’s HackerOne bug bounty program on September 8. Internal monitoring had not caught it. Cloudflare’s product security team reproduced the leak within 25 minutes using two separate accounts. By the evening of September 9, engineers had fixed the configuration, patched the engine, and restored public access.
Why a KV Cache Leak Matters for Hosted Inference
Shared inference works because providers pack many customers onto the same GPUs. That density keeps per-token prices low. However, it also places one tenant’s data in the same physical memory as everyone else’s. Isolation depends on the serving software tracking which memory belongs to which request, every single time.
This multi-tenant tradeoff is not new to AI. It mirrors the long-running debate over shared public cloud infrastructure versus dedicated hardware. What changes with LLMs is the payload. Prompts routinely carry source code, customer records, contracts, and internal documents that teams would never post to a shared disk.
Cloudflare says traffic patterns showed no evidence of deliberate exploitation. Still, the leaked fragments reached real customers making ordinary long-context calls. Nobody had to attack anything for data to cross the boundary.
How the KV Cache Leak Actually Worked
A KV cache stores the key and value tensors a transformer computes for each token. Reusing those tensors lets a model generate new tokens without reprocessing the whole prompt. To save time, SGLang does not zero cache pages between requests. Correct bookkeeping, rather than wiped memory, keeps one request from reading another’s data.
Two separate faults broke that bookkeeping. The first was a configuration mismatch. Cloudflare advertised a 1,048,576-token context window, but the prefill GPUs’ KV pool held only 681,280 tokens. SGLang calculates that pool at startup from whatever GPU memory remains after loading weights. No setting ties it to the context length.
The second fault was an engine bug in disaggregated mode, where separate GPU pools handle prefill and decode. SGLang replaced an over-length prompt with a single-token placeholder. However, the admission queue never marked that request as finished. The decode stage still allocated space for the entire prompt, then attended over leftover pages that earlier requests had written.
Cloudflare’s GLM-5.3 incident postmortem walks through the timeline and each upstream patch. In short, the fixes now retire rejected requests at admission, before any page allocation or KV transfer. Cloudflare also raised the prefill pool to roughly 2.04 million tokens, matching its GLM-5.2 setup.
How to Test Your Provider for a KV Cache Leak
Teams cannot inspect a provider’s GPU memory, but they can probe for obvious leakage. A simple canary test works against any OpenAI-compatible endpoint. Start by sending one request containing a unique random marker string. Next, send several unrelated requests from a fresh session. Finally, scan every response for the marker, since any hit means data crossed a request boundary.
Run the test after provider upgrades, against new endpoints, and in CI for self-hosted stacks. That said, a clean result proves little about edge cases. The GLM-5.3 bug fired only for prompts beyond the pool size on a misconfigured node. Black-box probing would rarely reach that state.
Treat this check like any other regression test in your pipeline. Teams already integrating AI into development workflows can add it beside existing evaluation suites. Automating it costs little and catches the obvious failure before a customer does.
Self-Hosting SGLang? Steps to Close the Gap
Operators running SGLang in disaggregated prefill and decode mode carry the same exposure. The engine fix shipped in SGLang v0.5.22, released October 19, so upgrade to that version or later. Then compare the KV pool size against the configured context length on every prefill and decode node. SGLang prints the pool size at startup, which makes the check easy to script.
Pool size shifts whenever weights, quantization, parallelism, or memory fraction change. Recheck after each change, and lower the context length if the pool falls short. Additionally, enable deferred KV release on both roles and cap Mooncake transfer batch sizes.
Where supported, turn on the --disaggregation-enable-kv-checksum and --kv-canary flags, then confirm verification in the logs. Those flags do not yet cover every architecture, including GLM-5.3 itself. Finally, stop proxies from aborting requests mid-transfer when clients disconnect, a change Cloudflare made to its own load balancer.
Tradeoffs and Common Mistakes
Zeroing every KV page between requests would close this class of bug outright. It would also erase much of the throughput that makes shared inference affordable. Most providers will instead layer checks on top, such as transfer checksums, per-page ownership tags, and canaries. Cloudflare says it plans to add page-level integrity tags and formally verify its RDMA transfer protocol.
The most common mistake is assuming a 200 response means the request succeeded. Nothing about the leaked outputs signaled failure. Another mistake is treating the advertised context window as a hardware guarantee. In this case, about 35 percent of the advertised window had no backing memory on the prefill nodes.
Teams handling regulated data should also ask vendors pointed questions. Which inference engine runs the model, and which version? How does the vendor verify tenant isolation after configuration changes? Dedicated or single-tenant deployments cost more, but they remove cross-customer exposure entirely.
Key Takeaways
- 155 GLM-5.3 responses on Workers AI exposed other customers’ data between August 27 and September 9, 2026.
- Only prompts longer than 681,280 tokens triggered the bug.
- A KV pool smaller than the context window combined with an SGLang disaggregation bug.
- SGLang v0.5.22 contains the engine fix.
- Canary tests catch gross leaks but cannot prove isolation.
Frequently Asked Questions About the Cloudflare KV Cache Leak
Which Cloudflare Models Did the KV Cache Leak Affect?
Only GLM-5.3 on Workers AI. Cloudflare reports no impact on any other model on the platform.
Did Anyone Exploit the KV Cache Leak?
Cloudflare investigators found no evidence of deliberate exploitation. Traffic matched ordinary long-context use after Cloudflare excluded its own testing and the researcher’s.
Do Workers AI Customers Need to Do Anything?
Cloudflare fixed the hosted deployment on September 9, so customers need to do nothing to stay protected. Any team that sent sensitive data to GLM-5.3 during the window should assess what it shared.
Which SGLang Version Fixes the Bug?
SGLang v0.5.22 includes the admission fix and related race-condition patches. Earlier hardening landed in v0.5.20 and v0.5.21.
What Is a KV Cache in an LLM?
It stores the attention keys and values computed for previous tokens. Reusing them lets the model generate each new token without recomputing the entire prompt.
Can a Canary Test Prove My Provider Is Safe?
No. It catches obvious cross-request leakage cheaply, but bugs triggered by size or timing can slip past it.
Final Thoughts
The Cloudflare KV cache leak came from two ordinary mistakes: a sizing mismatch and a missed status check. Neither looks dramatic alone. Together, they broke tenant isolation without a single error code. If your team sends sensitive context to hosted models, ask your provider how it verifies isolation after configuration changes. Self-hosting teams should upgrade SGLang to v0.5.22 and check pool size against context length today.
Photo by Hazel Z: Unsplash
A seasoned technology executive with a proven record of developing and executing innovative strategies to scale high-growth SaaS platforms and enterprise solutions. As a hands-on CTO and systems architect, he combines technical excellence with visionary leadership to drive organizational success.
