--- title: Vercel AI Gateway sidebar_label: Vercel AI Gateway sidebar_position: 48 description: Access language and embedding models through Vercel's unified AI Gateway --- # Vercel AI Gateway [Vercel AI Gateway](https://vercel.com/docs/ai-gateway) provides a unified interface to access models through a single API. This provider uses the official [Vercel AI SDK](https://ai-sdk.dev/). When [tracing](/docs/tracing/) is enabled, Promptfoo automatically turns on the AI SDK's built-in tracing for text generation, streaming, structured output, and embeddings. SDK spans inherit the current evaluation trace, including direct provider calls that supply a `traceparent`. Prompt and response content are not recorded. If you call the SDK directly from a [`file://` custom provider](/docs/providers/custom-api/), enable `experimental_telemetry` yourself; Promptfoo's [trajectory assertions](/docs/configuration/expected-outputs/deterministic/#trajectorytool-used) can normalize its tool-call spans from `ai.toolCall.name` plus the matching `ai.toolCall.args`, `ai.toolCall.arguments`, or `ai.toolCall.input` attributes. ## Setup 1. Enable AI Gateway in your [Vercel Dashboard](https://vercel.com/dashboard) 2. Get your API key from the AI Gateway settings 3. Set the `VERCEL_AI_GATEWAY_API_KEY` environment variable or specify `apiKey` in your config ```bash export VERCEL_AI_GATEWAY_API_KEY=your_api_key_here ``` ## Usage ### Provider Format The Vercel provider uses the format: `vercel:/` ```yaml providers: - vercel:openai/gpt-4o-mini - vercel:anthropic/claude-sonnet-5 - vercel:google/gemini-2.5-flash ``` ### Embedding Models For embedding models, use the `embedding:` prefix: ```yaml providers: - vercel:embedding:openai/text-embedding-3-small ``` ## Configuration ### Basic Configuration ```yaml providers: - id: vercel:openai/gpt-4o-mini config: temperature: 0.7 maxTokens: 1000 ``` ### Full Configuration Options ```yaml providers: - id: vercel:openai/gpt-4o-mini config: # Authentication apiKey: '{{env.VERCEL_AI_GATEWAY_API_KEY}}' # Or omit apiKey and set apiKeyEnvar: CUSTOM_API_KEY_VAR # Model settings temperature: 0.7 maxTokens: 2000 topP: 0.9 topK: 40 frequencyPenalty: 0.5 presencePenalty: 0.3 stopSequences: - "\n\n" # Request settings timeout: 60000 headers: Custom-Header: 'value' # Streaming streaming: true ``` When using Claude 5, omit `temperature`, `topP`, and `topK`. ### Configuration Parameters | Parameter | Type | Description | | ------------------ | -------- | ----------------------------------------------- | | `apiKey` | string | Vercel AI Gateway API key | | `apiKeyEnvar` | string | Custom environment variable name for API key | | `temperature` | number | Controls randomness; range depends on the model | | `maxTokens` | number | Maximum number of tokens to generate | | `maxRetries` | number | Retry attempts for a failed request | | `topP` | number | Nucleus sampling parameter | | `topK` | number | Top-k sampling parameter | | `frequencyPenalty` | number | Penalizes frequent tokens | | `presencePenalty` | number | Penalizes tokens based on presence | | `stopSequences` | string[] | Sequences where generation stops | | `timeout` | number | Request timeout in milliseconds | | `headers` | object | Additional HTTP headers | | `streaming` | boolean | Use the streaming API for text generation | | `responseSchema` | object | JSON schema for structured output | | `baseUrl` | string | Override the AI Gateway base URL | ## Structured Output Generate structured JSON output by providing a JSON schema: ```yaml title="promptfooconfig.yaml" # yaml-language-server: $schema=https://promptfoo.dev/config-schema.json providers: - id: vercel:openai/gpt-4o config: responseSchema: type: object properties: sentiment: type: string enum: [positive, negative, neutral] confidence: type: number keywords: type: array items: type: string required: - sentiment - confidence - keywords prompts: - 'Analyze the sentiment of this text: {{text}}' tests: - vars: text: 'I love this product!' assert: - type: javascript value: output.sentiment === 'positive' ``` ## Streaming Use Vercel's streaming API for text generation. Promptfoo collects the chunks and runs assertions on the completed response: ```yaml providers: - id: vercel:anthropic/claude-sonnet-5 config: streaming: true maxTokens: 2000 ``` The provider normalizes the AI SDK's `tool-calls` and `content-filter` finish reasons to `tool_calls` and `content_filter`. Use the [`finish-reason` assertion](/docs/configuration/expected-outputs/deterministic/#finish-reason) with these values, or `stop` and `length`, for both streaming and non-streaming responses. ## Supported Providers Query Vercel's public model catalog for IDs, capabilities, and pricing: ```bash curl -fsS https://ai-gateway.vercel.sh/v1/models ``` The endpoint requires no authentication. See the [Vercel AI Gateway documentation](https://vercel.com/docs/ai-gateway/models-and-providers) for response fields and filtering examples. ## Embedding Models Generate embeddings for text similarity, search, and RAG applications: Set the embedding provider for the `similar` assertion under `defaultTest.options.provider.embedding`: ```yaml title="promptfooconfig.yaml" providers: - vercel:openai/gpt-5.6-luna prompts: - 'Answer concisely: {{question}}' defaultTest: options: provider: embedding: id: vercel:embedding:openai/text-embedding-3-small tests: - vars: question: 'What is the capital of France?' assert: - type: similar value: Paris threshold: 0.8 ``` Supported embedding models: | Provider | Example Models | | -------- | ---------------------------------------------------------------- | | OpenAI | `openai/text-embedding-3-small`, `openai/text-embedding-3-large` | | Google | `google/gemini-embedding-001`, `google/text-embedding-005` | | Cohere | `cohere/embed-v4.0` | | Voyage | `voyage/voyage-3.5`, `voyage/voyage-code-3` | ## Examples ### Multi-Provider Comparison ```yaml title="promptfooconfig.yaml" # yaml-language-server: $schema=https://promptfoo.dev/config-schema.json providers: - id: vercel:openai/gpt-4o-mini config: temperature: 0.7 - id: vercel:anthropic/claude-sonnet-5 config: maxTokens: 1000 - id: vercel:google/gemini-2.5-flash config: temperature: 0.7 prompts: - 'Explain {{concept}} in simple terms' tests: - vars: concept: 'quantum computing' assert: - type: llm-rubric value: 'The response should be easy to understand' ``` ### JSON Response with Validation ```yaml title="promptfooconfig.yaml" # yaml-language-server: $schema=https://promptfoo.dev/config-schema.json providers: - id: vercel:openai/gpt-4o config: responseSchema: type: object properties: summary: type: string topics: type: array items: type: string wordCount: type: integer required: - summary - topics - wordCount prompts: - 'Analyze this article and return a structured summary: {{article}}' tests: - vars: article: 'Long article text...' assert: - type: javascript value: 'Array.isArray(output.topics) && output.topics.length > 0' ``` ## Environment Variables | Variable | Description | | ---------------------------- | ----------------------------------------------------------------------------- | | `VERCEL_AI_GATEWAY_API_KEY` | API key for AI Gateway | | `AI_GATEWAY_API_KEY` | Fallback from the shell environment when `VERCEL_AI_GATEWAY_API_KEY` is unset | | `VERCEL_AI_GATEWAY_BASE_URL` | Override the AI Gateway URL | ## Troubleshooting ### Common Issues 1. **Authentication Failed**: Ensure your `VERCEL_AI_GATEWAY_API_KEY` is set correctly 2. **Model Not Found**: Check that the provider/model combination is supported 3. **Request Timeout**: Increase the `timeout` configuration value ### Debug Mode Enable debug logging to see detailed request/response information: ```bash LOG_LEVEL=debug promptfoo eval ``` ## Related Links - [Vercel AI SDK Documentation](https://ai-sdk.dev/) - [Vercel AI Gateway](https://vercel.com/docs/ai-gateway) - [Supported Providers](https://vercel.com/docs/ai-gateway/models-and-providers) - [promptfoo Provider Guide](/docs/providers/)