Skip to content

Best practices

Your first request works. This group is about everything between that and traffic you can leave running unattended. Each page takes one problem that only shows up later — a prompt you cannot let a third party read, a response you cannot prove came from the model you paid for, a bill nobody capped, a provider that went down at the wrong moment.

Page When you need it What it solves
Private mode The prompt contains data a third party must not read in plaintext Routes the request only to a provider running the model inside an enclave, and fails rather than downgrading when none is free
Verify a response You have to show that the answer came from the model you asked for, not a cheaper substitute Checks the provider’s TEE signature, either through the Router or independently on your side
Cost caps A single call could cost more than you are willing to pay for it Bounds the price of a request before routing happens, and handles the empty-pool error that a tight ceiling produces
Failover & retries A provider is failing and you need to know whether the Router already handled it Explains the built-in cross-provider failover, when pinning silently disables it, and which remaining errors your client should retry
Production hardening You are about to point real traffic at the Router A pre-launch checklist for keys, resilience, spend, and explicit trust modes

Start at Production hardening if you want the whole list at once — it is the checklist that names every item below it, so it works as both the entry point and the last thing you tick off before launch. Otherwise pick by the problem in front of you. Private mode and response verification are the two things this platform does that a plain OpenAI-compatible endpoint cannot, so read them if either guarantee is why you are here. Cost caps and failover are not optional extras: every production integration ends up needing both, and it is cheaper to add them before the first incident than during one.

  • Trust modes — what standard, verified, and private each guarantee, before you enforce one.
  • Monitor balance and usage — the account-level alert that keeps a funded balance from reaching zero mid-shift.
  • Error codes — every status the Router returns and whether retrying it helps.