---
title: OpenAI GPT-Live
description: Test GPT-Live voice conversations with paced audio input, timestamped transcripts, recorded responses, backend delegation, and separate voice usage accounting.
---

# OpenAI GPT-Live

Use `openai:live:gpt-live-1` to evaluate [OpenAI's GPT-Live API](https://developers.openai.com/api/docs/guides/live). It connects to `/v1/live/sessions` and supports full-duplex audio, where the model can listen and speak simultaneously. `openai:gpt-live-1` and `openai:live` select the same provider, as do dated `openai:gpt-live-1-YYYY-MM-DD` snapshots. The official OpenAI endpoint requires a `gpt-live-*` model; compatible gateways can use custom model names. The transcription-only `gpt-live-transcribe` model requires a separate transcription session and is not supported here.

Set `OPENAI_API_KEY` to an OpenAI project key with Live access. For a compatible gateway, set `apiBaseUrl` and authenticate with `apiKey`, a credential header in `headers` (such as `Authorization` or `api-key`), or userinfo in `apiBaseUrl`, which is sent as a Basic `Authorization` header. When a gateway uses its own credential header or URL userinfo, an `OPENAI_API_KEY` from the environment is not sent to it unless the provider or prompt config sets `apiKey` or `apiKeyEnvar`. A prompt's `config` can set its own `apiBaseUrl`, `apiHost`, `organization`, and credentials; each session uses the merged endpoint, default headers, and credentials together. For gateways that authenticate with an opaque header name such as `X-Session-Access`, set `apiKeyRequired: false`; the header value is still redacted from diagnostics.

## Quickstart

A prompt that overrides the endpoint must supply its required credential headers in that prompt's `config.headers`. Credential headers from the provider configuration are not inherited by endpoint overrides.

```yaml title="promptfooconfig.yaml"
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
description: GPT-Live spoken answers
prompts:
  - 'What is the capital of France? Answer in one sentence.'
providers:
  - id: openai:live:gpt-live-1
    config:
      instructions: Keep answers short. Delegate questions needing research to the backend.
      audio:
        output:
          voice: marin
      responseWindowMs: 30000
      delegation:
        type: responses
        responses:
          model: gpt-6-luna
          instructions: Return concise, factual answers.
tests:
  - assert:
      - type: contains
        value: Paris
```

Run `npx promptfoo@latest eval --no-cache -o results.json`. Assertions evaluate the assistant's transcript. The result also includes playable audio and timestamped user and assistant transcript fragments. Fragments keep their original spacing, so a transcript can begin with a space; prefer `contains` or `icontains` to `equals`.

For native audio grading with `llm-rubric`, use an audio-capable Chat grader such as `openai:chat:gpt-audio-1.5`. GPT-Live used as a grader receives the transcript; it does not accept the MP3 and arbitrary WAV files supported by Chat audio grading.

Each test creates a new session. Text prompts seed a user message in startup history, then request a spoken answer while streaming silence. This is useful for checking answer content; use recorded audio to evaluate speech recognition, pauses, or interruptions.

## Audio input

Use OpenAI chat-format audio content in your prompt:

```json
[
  {
    "role": "user",
    "content": [
      {
        "type": "input_audio",
        "input_audio": { "data": "{{audio}}", "format": "wav" }
      }
    ]
  }
]
```

Set the `audio` test variable to `file://sample.wav`. WAV files must contain mono, signed 16-bit PCM matching `audio.format.rate` (24,000 Hz by default). The file must declare one valid format before its audio data, and each data chunk must contain complete PCM16 samples. Promptfoo removes the WAV container before streaming and ignores bytes after its declared RIFF boundary. It accepts streaming headers with unknown sizes, such as OpenAI text-to-speech `wav` output and ffmpeg pipe output, and `WAVE_FORMAT_EXTENSIBLE` PCM. It rejects mismatched rates and compressed formats; it does not resample audio. Base64 input may include or omit padding.

Raw base64 audio accepts `pcm16`, `g711_ulaw`, or `g711_alaw` in `input_audio.format`. Configure the corresponding shared input/output format:

| `audio.format`                   | Input encoding            |
| -------------------------------- | ------------------------- |
| `{type: audio/pcm, rate: 24000}` | `pcm16` or matching `wav` |
| `{type: audio/pcm, rate: 16000}` | `pcm16` or matching `wav` |
| `{type: audio/pcmu, rate: 8000}` | `g711_ulaw`               |
| `{type: audio/pcma, rate: 8000}` | `g711_alaw`               |

Audio is accepted only in the final user message. Supply prior history as text messages. System messages map to Live's developer role; user text stays user content. Images are not accepted by the Live voice frontend.

## Capture duration

Promptfoo streams audio in 20 ms frames at the configured sample rate. Recorded audio is followed by `responseWindowMs` of silence (default: 30 seconds). For text prompts, Promptfoo streams silence immediately, asks Live to answer, and starts the response window when Live acknowledges that instruction. It then sends `session.close` and waits for final usage. The input clip plus response window may total at most five minutes.

Live has no authoritative speech-completed event. The response window is a fixed recording window and may cut off speech. Increase it for long replies or backend work. Backend completion and transcript gaps do not end the capture early. Transcript and audio deltas received after closing begins are ignored; final usage and errors are still processed. Output audio contains the received samples; transcript timestamps are on the session timeline and do not establish playback timing.

Output audio is capped by the configured format and capture duration, with a maximum of five minutes. Startup delay does not increase this budget. Excess audio ends the capture with an error.

`websocketTimeout` covers the handshake, `session.started`, and a text prompt's acknowledgment (default: 30 seconds); `closeTimeoutMs` controls finalization (default: 15 seconds). These timeouts plus the capture duration must fit within `REQUEST_TIMEOUT_MS` (default: five minutes). Increase it for a full five-minute capture. Cancellation and eval shutdown release active sockets. Live responses are not cached.

## Backend delegation

For managed Responses delegation, set `delegation.type: responses` and `delegation.responses.model` to a nonempty string of at most 256 UTF-8 bytes. The backend can use `function` and `web_search` tools. It has its own instructions, token limit, reasoning settings, and service tier. Follow [OpenAI's delegation configuration](https://developers.openai.com/api/docs/guides/live-delegation) for supported settings.

For custom functions, set `functionCallHandler: file://tools.js`. Export an async function `(name, args, signal) => string`, where `args` is the JSON argument string. Promptfoo checks the function name against configured tools, collects completed calls, returns every result, and then continues the backend response. Handlers must enforce permissions for actions they execute.

`maxToolIterations` separately limits each session's function calls and distinct delegations, including client and managed Responses delegations (default: 8 each, allowed range 1 to 64). Repeated delegation IDs do not count again. When either limit is exceeded, Promptfoo ends the capture and reports an error without invoking another client handler. A managed Responses delegation may already be running when its creation event reaches Promptfoo.

Function-call IDs and names must be nonempty and are limited to 256 bytes each. Their combined size, including argument strings, is limited to 1 MiB across the session; exceeding this limit ends the capture before the call is buffered or executed.

Each function-handler result is limited to 1 MiB before it is serialized or sent. Return a summary or a reference for larger results.

For your own model or agent harness, use `delegation.type: client` and `delegationHandler: file://backend.js`:

```javascript
export default async function handleDelegation(request, signal) {
  // request: { id, offsetMs, input, transcript }
  // Consult your backend using the startup history and timestamped transcripts.
  // Honor signal to stop work when the eval ends.
  return 'The order shipped today.';
}
```

The delegation event contains an ID, not task text. Your handler receives a snapshot of the conversation collected so far. Return a concise string within Live's 500-token append limit. Promptfoo sends it as commentary using the original delegation ID. Handler errors are reported without sending exception details to the model. Late results are discarded after closing.

Promptfoo rejects client-handler results larger than 64 KiB before sending them. This bounds local buffering; the gateway still enforces the 500-token limit.

Conversation copies retained by pending client handlers are limited to 8 MiB of text and 50,000 input/transcript entries in total. Complete handlers promptly or reduce conversation history if this limit is reached. The budget is released when each handler settles, including after cancellation.

Omitting delegation selects client mode. If Live requests backend work without a handler, the eval reports an error. Backend work that is pending when the capture ends, or requested after it ends, also reports an error.

A spoken answer does not show that backend work ran. `metadata.delegations` lists each delegation's `id`, `target`, and `offsetMs`, and `metadata.backendResponses` lists completed Responses calls. Known credentials echoed in session IDs, backend metadata, or close reasons are redacted. Assert on these fields when a test requires delegation:

```yaml
assert:
  - type: javascript
    value: context.providerResponse.metadata.delegations.length > 0
```

## Errors and refusals

Startup failures report the server's error code and message. After startup, Live `error` events are recorded in `metadata.apiErrors` with the code, type, redacted message, parameter, and rejected client event, without ending the capture:

- Moderation and safety errors are graded as refusals, including when they reject one of promptfoo's commands or arrive while the session is closing.
- A rejected opening instruction or commentary ends the capture, because Live was never asked to speak.
- Other rejected commands and unattributed errors mark the result as an error and keep the full transcript.
- Errors for pending commands that `session.close` cancels are recorded but do not change the result.

Safety interventions are graded as refusals rather than errors. A moderation error sets `isRefusal` and `guardrails.flagged`. A session closed by the safety filter (`reason: content`) also sets `finishReason: content_filter` and `conversationEnded`, so multi-turn red team strategies stop probing. Sessions that close as `expired` or `connection_lost` remain errors.

## Results and cost

`output` is the assistant transcript, concatenated exactly as received. Transcripts preserve model content for grading, including sensitive content returned by the endpoint. `audio` contains playable PCM16 WAV at the session's sample rate, including decoded G.711 responses. `metadata.transcript` retains both speakers' fragments and `start_ms`/`end_ms` timestamps, including overlap. The request count includes the Live session and each completed backend response.

`metadata.voiceSeconds` is the latest valid cumulative usage snapshot. Usage must be finite, nonnegative, and no greater than `REQUEST_TIMEOUT_MS` rounded up to seconds. `metadata.finalUsageConfirmed` is true only when `session.closed` supplies valid final usage. A dropped connection preserves partial output and observed usage, and reports an error. Numeric overflow in voice, backend, or total cost reports an error and omits the affected cost.

Voice cost uses the published $0.05/minute rate for `gpt-live-1` and its dated snapshots, such as `gpt-live-1-2026-09-01`. Set `costPerMinute` to override it or to price other models. Responses token usage and model cost are accumulated separately from nested backend events. `cost` includes confirmed voice and backend model costs, excluding hosted-tool fees. Client-managed backend costs are unknown, so those sessions expose voice cost in `metadata.voiceCost` and omit total cost.

See the [runnable example](https://github.com/promptfoo/promptfoo/tree/main/examples/openai-live) and [OpenAI's Live session guide](https://developers.openai.com/api/docs/guides/live-conversations).