Your Agent Went Idle. Your Inference Bill Didn’t.

A coding agent finishes a task on Friday. On Tuesday, its user returns and asks it to pick up where it left off. To the user, that sounds like one continuing job. To the product, it may mean loading a large context, checking what changed, and paying to reconstruct a working state before any new work begins.

That gap between paused and cheap is becoming a pricing problem. Founders building long-running agents need to know what it costs to keep work resumable—and decide which part of that cost belongs on the invoice.

The expensive moment may be the restart

One developer describing heavy coding-agent use reported roughly 1.3 billion input tokens in a week against 7–8 million output tokens, with much of the input cached. The same account described idle sessions losing access to short-lived cloud caches, making a return to a large context expensive at API rates. This is one user’s account, not a workload benchmark, but it illustrates a real pricing exposure: input history can dominate the bill even when the agent produces little new text.^[1]

Consider a simplified example. Ten agents each need to revisit 500,000 tokens of prior context. That is five million input tokens before they do the next useful thing. At an illustrative, not quoted, rate of $2 per million uncached input tokens, resuming them costs $10; at $0.20 per million cached tokens, it costs $1. Repeated restarts can turn a small difference in cache behavior into a meaningful margin difference.

The lesson is not that every agent should keep a giant context alive. It is that a founder cannot forecast cost from completed tasks alone. Two customers can close the same number of issues while generating very different inference bills if one works in uninterrupted bursts and the other repeatedly resumes dormant projects.

An idle agent has more than one state

“Keep my agent ready” sounds like a single feature. Operationally, it can mean several different promises:

• Paused: Save task state and retrieve or reconstruct context when work resumes.
• Warm: Maintain enough reusable context to reduce the cost or latency of resumption.
• Ready: Reserve capacity and keep the agent close to immediate execution.

These states should not share a price merely because the interface shows the same agent avatar. Paused work may primarily incur storage and retrieval costs. Warm work can involve cache retention or context preparation. Ready work may consume scarce serving capacity even while the customer issues no requests. The implementation determines the cost; the label does not.

Nor does controlling your own inference make persistence free. Saving state to disk is different from keeping a model’s working context instantly reusable. A saved state may need to be loaded, revalidated, or recomputed. Documents and code can change while an agent sleeps. A cheap restart that confidently acts on stale information is not a bargain.

That is why a useful product metric is cost per successful resumption: the infrastructure and inference cost of returning an agent to a state where it can safely continue. Track it alongside resume latency and the share of resumed tasks that need substantial context rebuilt. Those measures expose a problem that a dashboard of tokens per request can miss.

Cheap tokens do not guarantee cheap work

Inference providers already treat cost-per-token economics and gross margin as product concerns. Mistral’s cloud and inference product-leadership listing explicitly connects pricing and packaging with gross margin, reliability, and cost-per-token economics.^[3] For an agent company buying that infrastructure, provider pricing is an input to the business model—not a customer-facing value proposition.

Raw token prices can also obscure differences in how much work a model takes to finish a task. A discussion of coding-model benchmarks points out that models with similar per-token prices can have substantially different total task costs because they consume different numbers of tokens.^[7] Add repeated context loading, and “we switched to a cheaper model” becomes an incomplete margin story.

Self-hosting changes the calculation rather than eliminating it. Inference cost depends on hardware, serving system, and traffic pattern; an older discussion of Llama deployment makes that dependence explicit.^[8] If your customers’ agents all wake at the start of the workday, capacity sized for the peak may sit underused overnight. If they wake unpredictably, latency promises become harder to meet. Utilization belongs in the resumption model too.

Price the promise, not the cache implementation

Customers should not need to understand cache-expiration windows to buy an agent. They do need to know what “come back later” means. A workable package could include paused projects with a stated resume-time target, then charge more for faster readiness or dedicated capacity where customers genuinely need it. The expensive promise is not that the agent remembers a name; it is that it can restart promptly with enough current context to act.

Keep the internal cost accounting more granular than the invoice. Measure uncached input, cached input, retrieval, storage, reserved capacity, and failed or repeated resumptions by customer and workflow. Then test whether the premium for faster resumption covers the incremental cost of delivering it. Do not turn every implementation detail into a billable line item.

There is a customer-trust boundary here. If a user pauses an agent and your cache expires, silently billing them for a full reread on return makes your infrastructure decision their surprise expense. Conversely, promising instant, unlimited resumption inside a flat fee leaves your business absorbing an unbounded obligation. A clear readiness level and a predictable included allowance are better than either surprise.

Find out what customers mean by “always on”

Before setting a price, watch actual return behavior. How long do projects sit idle? How often do users reopen them? What fraction of prior context is still relevant? Does a ten-second resume delay matter for an interactive coding session but not for an overnight audit? These questions separate valuable continuity from expensive theater.

The strongest pricing test is a choice: offer a lower-priced option that resumes after a short wait and a higher-priced option with a faster, more dependable restart. If customers consistently choose the latter for particular workflows, you have evidence that readiness carries value. If they do not, optimize the restart instead of packaging a premium nobody wants.

Agent products are often sold as labor that never stops. Their economics are subtler: work starts, waits, forgets, reloads, and starts again. Price the continuity customers can rely on, and engineer the idle time you can afford to provide.

References

1. JetBrains reports revenue growth, net financial loss for 2025 | Hacker News — https://news.ycombinator.com/item?id=49977072
2. VP Product, Cloud & Inference at Mistral | Andreessen Horowitz — https://jobs.a16z.com/jobs/mistral-ai/c55aa346-38ea-46b5-8b39-1ce40e56dfe1–vp-product-cloud-inference
3. Gemini 4 Argon | Hacker News — https://news.ycombinator.com/item?id=50073879
4. Does anyone know how to estimate the cost of inference using your own Llama2 mod… | Hacker News — https://news.ycombinator.com/item?id=36976041

Generated by AI · OpenAI · gpt-6-sol