OpenAI has overhauled prompt caching for its GPT-6 model family to give persistent agents higher default cache hit rates and new controls for developers who want to squeeze out more performance. The update, announced September 22, 2026, introduces a 30-minute reuse window for eligible shared prefixes, cache discounts of up to 90% on cached input tokens, and a set of monitoring and diagnostic tools that help teams see exactly where their prompts are hitting or missing the cache.
How the new caching system works
Persistent agents built on GPT-6 often make chains of API calls that carry forward the same instructions, tool definitions, and background context from earlier turns. The improved caching system automatically identifies that shared context and reuses the computation, which cuts response times and costs. Developers now get cache discounts whenever a qualifying prefix is reused within a 30-minute window, without needing to configure anything.
Mario Rodriguez, Chief Product Officer at GitHub, said the change has already reshaped Copilot's inference stack. "Over the past several months, we've reduced by more than 50% the share of prompt tokens requiring fresh processing across billions of requests to OpenAI models, relative to our previous baseline. The result is a more efficient inference stack and faster time to first response for developers."
Monitoring and diagnosing cache misses
A new Prompt Caching Dashboard shows the proportion of input served from cache and lets teams track hit rates over time. An input composition chart breaks down cached versus uncached tokens, making it easier to spot drops that coincide with application changes. When an unexpected miss occurs, a diagnostics tool compares the request against a recent response to identify what changed-model, tools, settings, or input-and estimates how many tokens were affected.
For teams running long-running agents, these tools have translated directly into cost savings. Arian Hanifi, CTO at an AI engineering firm, said the dashboard and diagnostics helped his team "improve cache hit rates by a few percentage points, reducing costs by 20%." His team now receives alerts when caching breaks and uses Codex agents to diagnose the root cause.
Explicit controls for fine-tuning cache behavior
Beyond the default improvements, developers can now place explicit cache breakpoints to mark which prompt prefixes should be reused. This is useful when some context remains stable across many requests while other parts change frequently. A refreshed prompt caching guide explains how breakpoints interact with tool definitions and input changes.
On GPT-6 models, reasoning effort can now be adjusted between responses without invalidating the cache. Developers append a configuration_update while leaving the request-level reasoning effort unchanged, so an agent can raise effort for a hard subtask or lower it for a routine follow-up without losing reusable context.
Tool management also affects cache persistence. OpenAI recommends keeping tool definitions, schemas, and ordering stable. Instead of removing unused tools-which would break the cache-developers can use allowed_tools to restrict which tools are callable or set tool_choice to none. New developer messages let teams append fresh instructions near the end of the context, overriding older ones without disturbing the cached prefix.
For latency-sensitive applications, cache prewarming prepares known context ahead of time-loading shared instructions, tool definitions, or reference material during startup so the model can respond immediately when a user's first request arrives.
Eugene Mikhantyev, an AI engineer, reported that moving session agents to explicit breakpoints raised cache hit rates on evaluations from 83% to 91% in under a week. "This meant fewer cache writes and lower inference costs with the same workload; the cache writes fell by roughly two-thirds, and inference costs by 36%." Bin Fan, an agent team lead, described a similar trajectory: working with OpenAI's engineering team, his group pushed cache hit rates from roughly 85% to consistently above 90% by refining breakpoint placement and combining explicit and automatic caching.
Why this matters for customer support, IT, development, and operations teams
Prompt caching is not a background optimization detail-it directly governs the cost and responsiveness of any application that uses persistent GPT-6 agents. For support and operations teams running agents that handle multi-turn conversations or long-running tasks, a few percentage points of cache hit rate can swing inference costs by 20% or more. The new dashboard and diagnostics tools put that visibility into the hands of the engineers maintaining these integrations, and the explicit breakpoint controls give them a lever to act on it without waiting for a model update. Teams that invest an afternoon in reviewing their prompt structure against the updated caching guide can expect measurable reductions in latency and spend.
Your membership also unlocks:
