What happens between your code and the model.

A request is checked and paid for before it reaches a provider, and settled once the response is complete. Nothing is sent upstream until your key, its rules and your balance have all passed.

1. 1

   Authenticate

   The key is hashed (SHA-256) and looked up; the raw key is never stored. Inactive keys, deactivated accounts and accounts with an unverified email are refused.
2. 2

   Apply the key's rules

   If the key has a model allowlist, the model id must be on it. If it has a monthly spend cap, this month's spend plus the request's estimated cost must fit under it.
3. 3

   Reserve the worst-case cost

   The most the request could cost is held from your balance, so concurrent requests can't overdraw it. See cost reservation.
4. 4

   Dispatch upstream

   Only the fields the API accepts are forwarded. Your request headers are not passed on.
5. 5

   Relay the response

   Non-streaming responses are returned as one JSON body; streams are relayed chunk by chunk.
6. 6

   Settle and log

   The real cost replaces the reservation, and a log row is written with the model, token counts, cost, latency and status. Prompt and completion text is not logged.

Send your key as Authorization: Bearer sk-ng-… (what every OpenAI SDK does) or as an X-API-Key header. Chat completions is the only generation endpoint: there are no embeddings, image or audio endpoints. Vision works through image parts in chat messages.

A model id is provider/model, for example anthropic/claude-sonnet-5.5 or openai/gpt-6-sol. The match is exact: no case folding and no guessing. An unknown or retired id returns 404.

The catalog syncs daily. Free (:free) variants and models without a fixed per-token price are never added, and a model whose upstream price becomes variable is taken offline rather than billed at a guess.

Before dispatch, the gateway estimates the most the request could cost and holds that amount. The hold is an atomic database update that only succeeds if your available balance covers it, so two requests can't both spend the same dollar.

Requests are forwarded to OpenRouter, which serves the model from one of the providers hosting it. Several providers often host the same open-weight model; choosing between them is OpenRouter's job, and it moves traffic away from a host that is failing. The model never changes: a request for one model is only ever served by that model.

Retries are safe because nothing is billed until a response completes. Streams aren't retried because tokens may already have reached you.

NeuronGate does not switch to a different model when one fails: a silent swap would change output quality, price and behaviour without telling you. If you want a fallback chain, keep it in your code where you can see and test it. Because every model uses the same API and the same key, it's a short loop:

from openai import OpenAI, APIStatusError, APIConnectionError

client = OpenAI(base_url="https://neurongate.net/v1", api_key="sk-ng-...")

# Your order of preference. NeuronGate never swaps models for you.
MODELS = ["~anthropic/claude-sonnet-latest", "~openai/gpt-latest", "deepseek/deepseek-v4-pro"]

def complete(messages):
    last_error = None
    for model in MODELS:
        try:
            return client.chat.completions.create(model=model, messages=messages)
        except APIConnectionError as e:
            last_error = e
        except APIStatusError as e:
            # 402 (balance/cap) and 4xx won't succeed on another model: stop.
            if e.status_code < 500 and e.status_code != 429:
                raise
            last_error = e
    raise last_error

With stream: true the response is server-sent events in the OpenAI chunk format. The gateway always asks the upstream for a final usage chunk, so token counts arrive even if you didn't set stream_options.include_usage. Every data: event is a standard chunk, so the OpenAI SDKs' usual chunk.choices[0] loop works as is:

: neurongate request_id=6957bbea-… model=~anthropic/claude-sonnet-latest adapter=openrouter

data: {"id":"gen-…","choices":[{"index":0,"delta":{"content":"Hel"}}]}

data: {"id":"gen-…","choices":[{"index":0,"delta":{"content":"lo"}}]}

data: {"id":"gen-…","choices":[{"index":0,"delta":{},"finish_reason":"stop"}],"usage":{"prompt_tokens":12,"completion_tokens":2,"total_tokens":14}}

data: [DONE]

• Gateway details (the request id, model and route) arrive as an SSE comment line starting with :, and keep-alives during long silences as : ping. SSE clients ignore comment lines. The request id is also in the X-Request-ID response header.
• An upstream failure after the stream has started arrives as an {"error": {...}} event followed by [DONE], because the HTTP 200 has already been sent.
• If you disconnect mid-stream, the tokens generated so far are still billed. The provider charges for them either way.

A request's charge is computed when it completes:

computed = input_tokens  × input_price  / 1,000,000
         + output_tokens × output_price / 1,000,000

charged  = max(computed, cost reported by the upstream)

The upstream's reported cost matters for models with prompt-length price tiers, where long prompts cost more than the listed base price. Arithmetic is done in exact decimals, not floating point. Failed requests cost nothing and their reservation is released.

Each request is logged with the model, input and output tokens, cost, latency, status and request id. That is what powers the logs and activity pages and the CSV export. The prompt and completion are not part of the log.

A network failure can leave you unsure whether a request ran. For non-streaming requests, send an Idempotency-Key header: a retry with the same key and the same body returns the original response instead of running (and billing) the request again.

curl https://neurongate.net/v1/chat/completions \
  -H "Authorization: Bearer sk-ng-..." \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: order-4821-summary" \
  -d '{"model":"deepseek/deepseek-v4-flash","messages":[{"role":"user","content":"Summarise order 4821"}]}'

• Reusing a key with a different body returns 409.
• The stored response is kept for 24 hours so retries can be answered, then deleted. Requests without the header store nothing.

Timeouts apply to silence, not total length: a long generation that keeps producing tokens isn't cut off. Need a higher request rate? Spread work across several keys, or get in touch.

Gateway errors use one JSON shape. detail is usually a string; for key-rule errors it is an object with a machine-readable type. Errors from the provider itself are passed through with their original status.

{
  "error": {
    "status_code": 402,
    "detail": "Insufficient balance",
    "request_id": "6b0d…"
  }
}