LLM Gateways

LLM gateways let external tools send model requests through Hatz instead of directly to an LLM provider. This is useful when a coding tool, automation platform, or internal app already supports an OpenAI-style or Anthropic-style API connection and lets you configure a custom base URL.

For endpoint schemas, parameters, and request examples, use the Hatz AI API docs.

What this does

Routes compatible API requests through Hatz.
Uses the requesting user's Hatz API key and normal Hatz model access.
Records successful generation usage against Hatz credits.
Keeps the external tool in charge of the user experience, tool loop, and follow-up calls.

What this does not do

It does not turn the external tool into Hatz Chat.
It does not automatically apply Hatz Chat memory, personalization, or the Hatz app harness.
It does not bypass model access, package limits, tenant restrictions, or rate limits.

Gateway formats

OpenAI Responses: Use /v1/openai/responses when the tool supports OpenAI Responses or an OpenAI SDK provider that can set a custom base URL. See the OpenAI Responses API docs.
Anthropic Messages: Use /v1/anthropic/messages when the tool expects Anthropic Messages format, such as Claude Desktop gateway configuration or Anthropic SDK clients. See the Anthropic Messages API docs.
Hatz-native chat: Use /v1/chat/completions when you want Hatz-native chat behavior, Hatz app orchestration, or Hatz-specific model routing. See the Chat Completions API docs.

Authentication

Create a Hatz API key from Workspace > API Keys. Use the same key in the external tool.

Most OpenAI-compatible tools expect:

Authorization: Bearer <Hatz API key>

Some clients also support:

X-API-Key: <Hatz API key>

Model IDs

Use model IDs from the Hatz model list, not display names. Query the available models with:

curl "https://ai.hatz.ai/v1/chat/models" \  -H "Authorization: Bearer $HATZ_API_KEY"

If a model is disabled, unavailable, or rate limited for the user, the gateway returns the same access outcome the user would get by requesting that model directly.

Tools and search

Gateway routes are client-managed for tools. The external tool asks the model for a tool call, executes the tool, and sends the tool result back in the next request.

For Anthropic-style web search requests, Hatz can emulate Anthropic's web search response shape with Hatz-hosted Firecrawl search when the feature is enabled for the environment. When search is not enabled, the gateway returns a compatible tool response saying search is unavailable instead of silently using a provider-owned search tool.

Rate limits and 429 errors

If a gateway request returns HTTP 429 Too Many Requests with a Retry-After header, the API key has reached a rate limit.

Availability: these limits are being rolled out gradually and may not apply to your organization yet.

Requests made with a Hatz API key are limited in three ways: requests per minute, tokens per minute, and requests running at the same time. This applies to:

/v1/openai/responses and /v1/anthropic/messages (including /v1/anthropic/v1/messages)
Chat, app and workflow requests: /v1/chat/completions, /v1/chat/platform/completions, /v1/app/{app_id}/query, /v1/app/{app_id}/query/platform and /v1/workflows/steps/query

All of these count toward the same limits, so a key that uses several of them shares one allowance across them. Other Hatz endpoints are not covered yet. Messages handled by the Hatz Slack and email integrations, and use of Hatz in your signed-in browser, are not counted.

Each API key has its own limits. The user and the organization that own the key also have limits, shared by all of their keys. The strictest limit that applies is the one that counts, so traffic from other keys in your organization can cause a 429 on yours.
Requests and tokens per minute are counted per model, so a busy model does not use up the allowance for another. The limit on requests running at the same time applies across all models, and a streaming request counts as running until its stream ends.
Tokens are estimated from your input when the request arrives, then corrected with the actual usage when it finishes.

When a key is over one of its limits:

The request is refused with HTTP 429 before its response is generated.
Hatz does not retry it or send it to another model.
Streaming requests get the same JSON error; no event stream is started.

Wait for the time in the Retry-After header, then send the request again. If you hit limits often, lower how many requests your client sends at once and back off between retries. The response includes:

Retry-After: seconds to wait, rounded up.
retry-after-ms: the same wait in milliseconds.
X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset: the limit that was reached, what is left (0), and seconds until it resets. These are sent only when a per-minute limit (requests or tokens) was reached, not when too many requests are running at once.

The error body names which limit was reached: key_rate_limited (this key), user_rate_limited (all keys of the user who owns it), or tenant_rate_limited (all keys in your organization).

OpenAI Responses: error.type is rate_limit_error and error.code is the limit name.
Anthropic Messages: error.type is rate_limit_error and error.hatz_rate_limit has code and retry_after_seconds.
Chat, app and workflow endpoints (including /v1/chat/completions): error.type is rate_limit_error, error.code is the limit name, and error.details.retry_after_seconds is the wait in seconds. This is not the OpenAI Responses error shape.

Check the code, not the message text: the message always says "this API key", even when the user or organization limit was reached.

These codes mean "slow down and retry". A request refused because of credits or usage limits does not use them (on Anthropic Messages it carries error.hatz_usage_limit instead), and retrying will not help until credits or the usage limit are restored.

If you receive one of these errors and did not expect a limit, or need higher limits, contact Support with the error code and the request ID from the response.

Usage and credits

Successful generation requests create Hatz usage records. Token counting or preflight endpoints are used for limit checks and do not create usage records by themselves. Usage accounting is based on the final model response usage fields and internal Hatz accounting metadata.

Troubleshooting

If a gateway request fails:

Confirm the Hatz API key is valid and belongs to the intended user.
Confirm the external tool is using the correct base URL for the selected API format.
Confirm the model ID appears in /v1/chat/models for that user's tenant.
Check whether the selected model is rate limited or disabled for the tenant, role, or package.
For a 429 with key_rate_limited, user_rate_limited, or tenant_rate_limited, see Rate limits and 429 errors above.
Capture the request ID from the error response or logs when contacting support.