Skip to main content

LLM Gateways

LLM Gateways

LLM gateways let external tools send model requests through Hatz instead of directly to an LLM provider. This is useful when a coding tool, automation platform, or internal app already supports an OpenAI-style or Anthropic-style API connection and lets you configure a custom base URL.

For endpoint schemas, parameters, and request examples, use the Hatz AI API docs.

What this does

  • Routes compatible API requests through Hatz.

  • Uses the requesting user's Hatz API key and normal Hatz model access.

  • Records successful generation usage against Hatz credits.

  • Keeps the external tool in charge of the user experience, tool loop, and follow-up calls.

What this does not do

  • It does not turn the external tool into Hatz Chat.

  • It does not automatically apply Hatz Chat memory, personalization, or the Hatz app harness.

  • It does not bypass model access, package limits, tenant restrictions, or rate limits.

Gateway formats

  • OpenAI Responses: Use /v1/openai/responses when the tool supports OpenAI Responses or an OpenAI SDK provider that can set a custom base URL. See the OpenAI Responses API docs.

  • Anthropic Messages: Use /v1/anthropic/messages when the tool expects Anthropic Messages format, such as Claude Desktop gateway configuration or Anthropic SDK clients. See the Anthropic Messages API docs.

  • Hatz-native chat: Use /v1/chat/completions when you want Hatz-native chat behavior, Hatz app orchestration, or Hatz-specific model routing. See the Chat Completions API docs.

Authentication

Create a Hatz API key from Workspace > API Keys. Use the same key in the external tool.

Most OpenAI-compatible tools expect:

Authorization: Bearer <Hatz API key>

Some clients also support:

X-API-Key: <Hatz API key>

Model IDs

Use model IDs from the Hatz model list, not display names. Query the available models with:

curl "https://ai.hatz.ai/v1/chat/models" \
  -H "Authorization: Bearer $HATZ_API_KEY"

If a model is disabled, unavailable, or rate limited for the user, the gateway returns the same access outcome the user would get by requesting that model directly.

Tools and search

Gateway routes are client-managed for tools. The external tool asks the model for a tool call, executes the tool, and sends the tool result back in the next request.

For Anthropic-style web search requests, Hatz can emulate Anthropic's web search response shape with Hatz-hosted Firecrawl search when the feature is enabled for the environment. When search is not enabled, the gateway returns a compatible tool response saying search is unavailable instead of silently using a provider-owned search tool.

Usage and credits

Successful generation requests create Hatz usage records. Token counting or preflight endpoints are used for limit checks and do not create usage records by themselves. Usage accounting is based on the final model response usage fields and internal Hatz accounting metadata.

Troubleshooting

If a gateway request fails:

  • Confirm the Hatz API key is valid and belongs to the intended user.

  • Confirm the external tool is using the correct base URL for the selected API format.

  • Confirm the model ID appears in /v1/chat/models for that user's tenant.

  • Check whether the selected model is rate limited or disabled for the tenant, role, or package.

  • Capture the request ID from the error response or logs when contacting support.

Did this answer your question?