Data extraction

Documents arrive in every layout imaginable, and rule-based parsers break on the first new template. Models read them well but occasionally return a malformed date, a missing field or a total that doesn't add up. Extraction is only useful if bad output is caught before it reaches your database.

Vision-capable models accept images as image_url content parts, including base64 data URLs, so a scanned invoice can go straight to the model. Use the catalog's vision filter to see which models take images.

A common pattern: run everything through the cheap model, and send only documents that fail validation twice to the stronger one. Current prices are in the model catalog.

The schema is the contract. The model is told the fields; your code checks them and retries once with the validation error so the model can correct itself.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://neurongate.net/v1",
    api_key=os.environ["NEURONGATE_API_KEY"],
)
import base64, json
from datetime import date
from pydantic import BaseModel, ValidationError

class Invoice(BaseModel):
    vendor: str
    invoice_number: str
    issue_date: date
    currency: str
    total: float

PROMPT = (
    "Extract the invoice as JSON with keys vendor, invoice_number, "
    "issue_date (YYYY-MM-DD), currency (ISO code) and total (number). "
    "Reply with the JSON object only."
)

def extract(image_path: str) -> Invoice:
    b64 = base64.b64encode(open(image_path, "rb").read()).decode()
    messages = [{"role": "user", "content": [
        {"type": "text", "text": PROMPT},
        {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"}},
    ]}]
    for attempt in range(2):
        reply = client.chat.completions.create(
            model="~google/gemini-flash-latest", messages=messages, temperature=0,
        )
        raw = reply.choices[0].message.content or ""
        try:
            text = raw.strip().removeprefix("```json").removesuffix("```")
            return Invoice.model_validate(json.loads(text))
        except (json.JSONDecodeError, ValidationError) as err:
            messages += [
                {"role": "assistant", "content": raw},
                {"role": "user", "content": f"That was invalid: {err}. Reply with corrected JSON only."},
            ]
    raise ValueError(f"Could not extract {image_path}; flag for review")

1. 1

   Give the workload its own key

   Create a key just for the extraction pipeline under API keys. Its usage then shows up separately in the logs, and revoking it touches nothing else.
2. 2

   Set a monthly spend cap

   Once the key has spent its cap for the month, its requests are refused until the next month or until you raise the cap. Size it from a week of real traffic, not a guess.
3. 3

   Allowlist the models it may call

   Restrict the key to the models this workload actually uses, for example ~google/gemini-flash-latest and ~anthropic/claude-sonnet-latest. A typo or a prompt-injected model switch then fails instead of quietly running on a pricier model.
4. 4

   Watch the logs

   Filter the request log by this key to see tokens, cost and latency per call. Export CSV for a monthly cost-per-workload report.