Tutorial: Budget for Prompt-Length Price Tiers

A growing number of models price a request by its prompt length: below a threshold you pay one rate, above it you pay a higher rate. The higher rate typically applies to the whole request, not just the tokens past the line. Claude Haiku 5.5 is a clean example:

A 99K-token prompt with a 1K-token answer costs about $0.0104. Add 2K tokens of retrieved context and the same request costs about $0.0530, five times as much, for 2% more input.

Tiers like this are easy to miss because catalogs, dashboards and price comparison tables usually show one number per model: the base rate.

Step 1: Find the tiered models you use

For each model in production, check the provider's pricing page for a threshold. Record it next to the base price:

anthropic/claude-haiku-5.5:
  base:      { input: 0.10, output: 0.50 }
  threshold: 100000            # prompt tokens
  above:     { input: 0.50, output: 2.50 }

Long-context surcharges are common on models with very large windows, so check every model you use for prompts over roughly 100K tokens.

Step 2: Estimate with the tier, not the base rate

def estimate_cost(prompt_tokens: int, output_tokens: int, price: dict) -> float:
    tier = price["above"] if prompt_tokens > price.get("threshold", float("inf")) else price["base"]
    return (prompt_tokens * tier["input"] + output_tokens * tier["output"]) / 1_000_000

Use this in budget forecasts and pre-flight checks. A forecast that multiplies tokens by the base rate under-counts every long request.

Step 3: Keep requests away from the line

Most tier jumps are accidental: retrieval adds a few more chunks than usual, or a conversation grows past the threshold. Three guards:

1. Cap retrieved context so the full prompt stays a safe margin under the threshold, say 90% of it.
2. Summarize long histories before they cross the line, instead of after.
3. Route intentionally long prompts elsewhere if a different model is cheaper above the threshold for the same quality.

Step 4: Alert on tier crossings

Log prompt tokens per request and count how many land above each model's threshold. A sudden rise usually means a retrieval or prompt-template change, and it shows up in the tier count days before it shows up in the invoice.

Step 5: Reconcile against what you were charged

Compare your estimate with the provider's reported cost per request. If they diverge on long prompts, your threshold or tier prices are wrong.

NeuronGate angle

NeuronGate shows one base price per model in the model catalog. When the upstream provider reports the actual cost of a request, NeuronGate bills whichever is higher, that or the base price times your tokens, so long prompts are billed at their real tier. Your request logs in usage carry prompt and completion tokens for every call, which is all Step 4 needs.

FAQ

Does the higher rate apply to the whole request?

For the tiered models described here, the tier is chosen by the prompt's length and applies to that request's input and output. Check each provider's pricing page for its exact rule.

Is a tiered model still worth using for long prompts?

Often, yes. Haiku 5.5 above 100K tokens is still half the price of Haiku 4.5. Just budget with the right rate.

Sources and context

• Anthropic: Claude Haiku 5.5 pricing