Production checklist

Ship a compatible endpoint as a tested infrastructure dependency, not as an interchangeable URL.

Configuration

• Keep baseUrl, credentials, headers, adapter, and model ID in server-side configuration.
• Validate required variables and the endpoint URL at startup.
• Use HTTPS outside intentional local development.
• Pin the model factory's api alongside its modelId.
• Allow-list model IDs when users or tenants can choose a model.
• Treat endpoint, adapter, model, and important providerOptions as one release unit.

Capability gates

• Run direct and streaming completion tests.
• Test required tools, streamed arguments, and multi-turn tool replay.
• Validate structured output locally before product state depends on it.
• Test representative images, documents, embeddings, audio, and transcription separately when used.
• Confirm usage normalization before using provider counts for quotas or billing.
• Do not treat /models as capability discovery.

Reliability

• Bound timeouts, concurrency, and retries at the application boundary.
• Retry only failures known to be transient for the selected provider.
• Make downstream tool writes idempotent before retrying an agent run.
• Rate-limit by tenant before traffic reaches a shared gateway.
• Avoid silent adapter, endpoint, or model fallback.
• Re-run the compatibility suite after endpoint, model, gateway, or package changes.

Security and data

• Never expose provider keys or privileged gateway headers to browser code.
• Do not let model output choose an arbitrary baseUrl, credential, or routing header.
• Restrict outbound endpoints when configuration varies by tenant.
• Remove secrets and unnecessary personal data before requests.
• Apply an explicit retention policy to prompts, media, reasoning metadata, and tool results.
• Return application-safe errors rather than raw provider payloads.

Observability

Record the provider, endpoint identity, model ID, adapter, latency, normalized usage, status code, error class, and application request ID. Do not log credentials, full authorization headers, or unredacted payloads.

Attach Lens, Langfuse, or another Anvia observer when runs need model and tool traces. Choose safe or full payload capture deliberately; a compatible gateway can receive the same sensitive content as a first-party provider.

Rollout

Introduce a new compatible deployment behind controlled routing. Compare quality, latency, tool accuracy, schema validity, and cost with the current model before increasing traffic.

Keep rollback explicit: switch to a separately configured and separately tested model boundary rather than mutating a failed request in the middle of a run.