rate_limit_exceeded, with Retry-After and X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset
502
upstream_error
backend_unavailable, backend_error
503
gatekeeper_error
attestation_failed, attestation_revoked
500
server_error
internal
The 401 the production router answers to a missing key, verbatim:
json
{"error":{"message":"Incorrect API key provided. Send it as \"Authorization: Bearer sk-tee-v1-…\".","type":"authentication_error","code":"invalid_api_key"}}
gatekeeper_error
A 503 with type: gatekeeper_error comes from one of two verifiers, and the body says which through stage:
Your Gatekeeper, when the endpoint it fronts has no valid verdict. The body carries stage and reason beside error; see Read a verdict.
The router, for a model served by an external endpoint. Here the router is a Gatekeeper toward that upstream and uses the same vocabulary. attestation_failed means nothing was sent: no live verdict admits the endpoint, because it is pending or denied. attestation_revoked means the request was admitted and then the verdict was withdrawn while the generation was streaming; the stream ends with this error as its last event, the generation is metered aborted with the tokens already delivered, and a client should not retry until an administrator acts.
A model whose endpoint holds no verdict is absent from GET /v1/models and answers 404 on GET /v1/models/{id}.
Rate limits
Four one-minute buckets, checked on admission in this order: requests per key, requests per workspace, tokens per key, tokens per workspace.
The per-key values come from the key where it sets them (Requests per minute, Tokens per minute on the key) and from the deployment's defaults otherwise. The workspace buckets always use the deployment's defaults, so minting more keys does not buy a workspace more budget. The production deployment's defaults are 600 requests and 2,000,000 tokens per minute.
Token cost is unknown until the model answers, so the token buckets are checked for being empty on admission and debited with real usage afterwards. A request may overshoot by its own size, never by more.
Past the limit the answer is 429 with Retry-After. The invitation lookup, the sign-in code requests and the analytics ingest have budgets of their own, so a workspace's traffic and the public lookups never spend each other's.
Credit
Credit is prepaid, in dollars, per workspace. A request that would take the balance below zero answers 402 insufficient_credits; one in-flight generation may push the balance negative by its own cost, no more (allowOverdraftMicros is 0 on production). A key with a Spend limit answers 402 key_spend_limit_reached once its own usage reaches it, whatever the workspace balance.
Where credit comes from and how it is shown: Credits.
Context length
A prompt longer than the model's context window answers 400 context_length_exceeded. The window is on the Models screen and in context_length on GET /v1/models.