For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

Configure agentgateway for providers without built-in support that implement the OpenAI API format.

Use this page for providers that implement the OpenAI API format but do not have a first-class provider: support yet. For built-in providers such as Baseten, Cerebras, Cohere, DeepInfra, DeepSeek, Fireworks AI, Groq, Hugging Face, Mistral, OpenRouter, Together AI, xAI, and Ollama, use the dedicated provider pages instead.

Before you begin

You also need the following prerequisites.

• An API key for your chosen provider, unless you are pointing to a local endpoint such as vLLM or LM Studio.

Configuring a custom provider

With a custom provider, you provide the API endpoint and a list of formats it supports. Agentgateway will automatically handle mapping between the incoming format and the supported formats.

The formats list decides which conversion an incoming request takes, and the conversions do not all carry the same feature set. A Messages request to a provider that declares responses takes the Responses conversion, even when the provider also declares completions, and that conversion drops extended-thinking history without an error. To carry thinking history across turns, declare completions and not responses, or set the AGENTGATEWAY_MESSAGES_PREFER_COMPLETIONS environment variable to true. For what each conversion keeps and drops, see Provider format conversion.

The formats list is optional. A model without it accepts only requests on paths that are forwarded to the provider without format conversion, such as /v1/systemone, /v1/ocr, /v1/images/generations, and /v1/responses/compact. A request in an LLM API format, such as a chat completions or messages request, has no format to convert to and is rejected. For an example, see the Jev guardrail guide.

Below shows an example of connecting to Perplexity, which exposes an OpenAI-compatible API for search-augmented models and does not currently have a first-class provider.

cat > /tmp/test-perplexity.yaml << 'EOF'
# yaml-language-server: $schema=https://agentgateway.dev/schema/config
llm:
  models:
  - name: "*"
    provider:
      custom:
        formats:
          # Indicate this provider supports the completions API. With no `path` specified, this defaults to <baseUrl>/chat/completions
          - type: completions
          # Indicate this provider supports the messages API, on a custom path /messages-api
          # - type: messages
          #   path: /messages-api
          # All possible APIs:
          # - type: embeddings
          # - type: responses
          # - type: realtime
          # - type: anthropicTokenCount
          # - type: generateContent
          # - type: geminiCountTokens
          # - type: rerank
    params:
      apiKey: "$PERPLEXITY_API_KEY"
      model: llama-3.1-sonar-large-128k-online
      baseUrl: "https://api.perplexity.ai"
EOF