Caching and reuse

What Sandbox can make faster, what model providers cache, and why pass@k attempts stay fresh.

The safe rule is simple:

Cache the box, not the grade.

Sandbox can avoid rebuilding images, keep capacity ready, and reuse one workspace for an interactive session. It must not reuse an answer or reward between benchmark attempts.

What is cached

Layer	Owner	Reused	Does it change the answer?
Task image	Sandbox BuildKit / Artifact Registry	Docker layers and the built task image	No
Node capacity	Sandbox warm placeholders	Room on a gVisor node	No
Ready CodeEdit pod	Sandbox claimable pool	A clean execution pod	No
Long-lived workspace	Sandbox session	Files and installed tools within one caller-owned session	Yes, intentionally: it is one continuing session
Prompt prefix / KV state	Model provider	Repeated system prompt, tools, and conversation prefix	No; the provider still generates a new response
Pass@k answer or reward	Nobody	Never reused	Reuse would invalidate the evaluation

Provider prompt caching

Prompt caching happens at the model provider, not inside Sandbox. Sandbox does not keep a cross-job LLM KV store.

OpenAI / Codex: eligible prompts are cached automatically. Cache hits require a long, identical prefix. Keep system instructions and tool definitions stable and append new turns instead of rewriting earlier ones.
Anthropic / Claude Code: Claude Code manages prompt caching for its own agent loop. Changing the model, tools, or reasoning effort changes the prefix and can lose the cache.
Other Harbor agents: use only kwargs documented by that agent. The platform forwards agent_options.agent_kwargs (Agent Runtime) and agents[].params / harbor_extensions.agent_kwargs (Harbor) as Harbor --ak values, but forwarding a field does not make an unsupported CLI honor it.

Harbor's verified opt-in example is Aider:

{
  "agents": [
    {
      "name": "aider-cached",
      "harbor_agent": "aider",
      "model": "anthropic/claude-sonnet-4-5",
      "params": {"cache_prompts": true}
    }
  ]
}

Harbor maps cache_prompts to Aider's --cache-prompts. Claude Code and Codex manage eligible caching automatically; they do not expose cache_control or prompt_cache_key as Harbor agent kwargs. Harbor also recognizes DISABLE_PROMPT_CACHING=1 only for Claude Code's Bedrock operator path; it disables caching and is not an integrator speed knob.

For example, reasoning effort is a supported agent kwarg:

{
  "agent_id": "codex",
  "model": "openai/gpt-5.5",
  "instruction": "Fix the failing tests",
  "agent_options": {
    "agent_kwargs": {
      "reasoning_effort": "xhigh"
    }
  }
}

This does not enable a Sandbox cache. Codex and OpenAI decide whether the stable prefix is eligible for a provider-side cache hit. Inspect n_cache_tokens and cost_usd in the run telemetry instead of assuming a hit.

Do not pass raw provider request fields blindly

A field such as cache_control or prompt_cache_key is valid only when the selected agent constructor explicitly supports it. Agent kwargs are not a generic tunnel into every provider request. Unsupported kwargs fail the run.

Reasoning effort is not cacheable work

reasoning_effort=high, xhigh, or max asks the model to spend more time and tokens reasoning. Sandbox can keep queue and startup time low, but it cannot skip those reasoning tokens while still claiming the same effort level.

Measure the two parts separately:

Timing	Controlled by
`queue_ms + startup_ms`	Sandbox capacity, images, scheduler
`agent_execution`	Agent harness, model, prompt, reasoning effort

See Cost model for the compute/model split.

Pass@k stays fresh

For a benchmark, submit one Harbor job with pass_at_k. Leave n_concurrent unset unless you need a tighter cap — the platform defaults it to min(pass_at_k, 16) so pass@8 fans out with eight concurrent trials:

{
  "task_slug": "my-task",
  "agents": [
    {
      "name": "codex-pass8",
      "harbor_agent": "codex",
      "model": "openai/gpt-5.5",
      "pass_at_k": 8,
      "harbor_extensions": {
        "agent_kwargs": {"reasoning_effort": "xhigh"}
      }
    }
  ]
}

Set "n_concurrent": 4 only when you deliberately want a lower fan-out.

Each attempt gets a fresh trial and verifier result. It may benefit from the same cached task image, warm node capacity, and provider-managed prompt prefix, but never from another attempt's answer, workspace changes, or reward.

Read result.outcome.trials_detail to see which attempts completed vs errored. Incomplete attempts are refilled with POST /sandbox/harbor/v2/jobs/{job_id}/retry-errored-trials (batch refill of every errored row — not per-seat patching). See Harbor datapoint.

Do not implement pass@8 by opening eight independent Agent Runtime jobs unless you deliberately want eight queue entries and have sized the worker/admission ceiling for them.

Reuse one sandbox for interactive work

If the work is a continuing loop rather than a benchmark, use a long-lived sandbox. Create once, run many commands against the same filesystem, then stop or destroy it.

Do not reuse that session across pass@k attempts. Interactive continuity and independent evaluation are different contracts.