Content generation

Writing 5,000 product descriptions by hand takes weeks; generating them with the most capable model is fast but expensive, and fanning out thousands of requests at once trips rate limits. You also need the output to sound like your brand, not like a generic model.

Every pass uses the same endpoint and key, so switching the draft model is a one-line change and you can compare two models on the same rows before committing to one.

Current prices per million tokens are in the model catalog.

Run requests in parallel, but only a fixed number at a time. It keeps throughput high without a burst of thousands of simultaneous requests, and a failure on one row doesn't stop the batch.

import asyncio, os
from openai import AsyncOpenAI

client = AsyncOpenAI(
    base_url="https://neurongate.net/v1",
    api_key=os.environ["NEURONGATE_API_KEY"],
)
STYLE = "Write in second person, under 80 words, no superlatives."
limit = asyncio.Semaphore(8)  # at most 8 requests in flight

async def describe(product: dict) -> str | None:
    async with limit:
        try:
            reply = await client.chat.completions.create(
                model="deepseek/deepseek-v4-flash",
                max_tokens=200,
                messages=[
                    {"role": "system", "content": STYLE},
                    {"role": "user", "content": f"Product: {product}"},
                ],
            )
            return reply.choices[0].message.content
        except Exception as err:
            print("failed", product["sku"], err)
            return None  # retry these later instead of failing the batch

async def main(products: list[dict]):
    return await asyncio.gather(*(describe(p) for p in products))

Set max_tokens on every request; it is the simplest bound on what a batch can cost. A balance can't be overdrawn: if it runs out mid-batch, the remaining requests are refused, so top up before large runs.

1. 1

   Give the workload its own key

   Create a key just for content batches under API keys. Its usage then shows up separately in the logs, and revoking it touches nothing else.
2. 2

   Set a monthly spend cap

   Once the key has spent its cap for the month, its requests are refused until the next month or until you raise the cap. Size it from a week of real traffic, not a guess.
3. 3

   Allowlist the models it may call

   Restrict the key to the models this workload actually uses, for example deepseek/deepseek-v4-flash and ~anthropic/claude-sonnet-latest. A typo or a prompt-injected model switch then fails instead of quietly running on a pricier model.
4. 4

   Watch the logs

   Filter the request log by this key to see tokens, cost and latency per call. Export CSV for a monthly cost-per-workload report.