# integration-opentelemetry/python (Python OpenTelemetry Tracing Example)

This example demonstrates how to use OpenTelemetry with Python to trace the internal operations of your LLM providers during Promptfoo evaluations. It uses the **protobuf format** for trace export, which is the default and most efficient format for the Python OpenTelemetry SDK.

Requires Python 3.10+ and Node.js >=22.22.0.

## Quick Start

On Windows (PowerShell), use `npx.cmd` instead of `npx` for the Promptfoo commands in this guide.

```bash
npx promptfoo@latest init --example integration-opentelemetry/python
cd integration-opentelemetry/python
```

On macOS/Linux, install the dependencies in a virtual environment:

```bash
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt

```

On Windows (PowerShell):

```powershell
python -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements.txt
$env:PROMPTFOO_PYTHON = (Resolve-Path .\.venv\Scripts\python.exe).Path
```

Run the evaluation using the configured virtual environment:

```bash
npx promptfoo@latest eval --no-cache
npx promptfoo@latest view
```

## Environment Variables

This example requires no API keys - it uses a simulated provider that demonstrates tracing patterns.

## Overview

This example showcases:

- **Python OpenTelemetry SDK** - Using the official Python SDK for tracing
- **Protobuf format** - The `opentelemetry-exporter-otlp-proto-http` package sends traces in protobuf format (`application/x-protobuf`), which is more efficient than JSON
- **Distributed tracing** - Parsing W3C Trace Context from Promptfoo and creating child spans
- **Trace assertions** - Validating trace structure and performance

## How It Works

1. **Promptfoo starts the OTLP receiver** on port 4318
2. **Promptfoo generates a trace context** for each test case (W3C Trace Context format)
3. **The Python provider receives the trace context** via `promptfoo_context['traceparent']`
4. **The provider creates child spans** using the OpenTelemetry Python SDK
5. **Traces are exported in protobuf format** to Promptfoo's OTLP endpoint
6. **Promptfoo correlates traces** with test cases for analysis

## Files in This Example

| File                   | Description                                        |
| ---------------------- | -------------------------------------------------- |
| `promptfooconfig.yaml` | Evaluation config with tracing enabled             |
| `provider.py`          | Python provider with OpenTelemetry instrumentation |
| `requirements.txt`     | Python dependencies (OpenTelemetry SDK)            |

## Protobuf export

The `opentelemetry-exporter-otlp-proto-http` package sends OTLP protobuf over HTTP.
Promptfoo also accepts OTLP JSON from other clients; the Python example uses
protobuf throughout.

## Provider Implementation

The key parts of the Python provider:

### 1. Initialize OpenTelemetry

```python
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import SimpleSpanProcessor

resource = Resource.create({
    "service.name": "my-python-provider",
    "service.version": "1.0.0",
})

exporter = OTLPSpanExporter(
    endpoint="http://localhost:4318/v1/traces",
)

# Use SimpleSpanProcessor for synchronous export
# This ensures spans are exported before the provider returns
provider = TracerProvider(resource=resource)
provider.add_span_processor(SimpleSpanProcessor(exporter))
trace.set_tracer_provider(provider)

tracer = trace.get_tracer("my-python-provider")
```

> **Note:** This example uses `SimpleSpanProcessor` for synchronous, immediate export. This ensures spans are sent before the provider returns. For production use with higher throughput, consider `BatchSpanProcessor`, but be sure to call `processor.force_flush()` before returning from your provider.

### 2. Parse Trace Context

```python
import re
from opentelemetry.trace import SpanContext, TraceFlags

def parse_traceparent(traceparent: str) -> SpanContext | None:
    match = re.match(r"^(\d{2})-([a-f0-9]{32})-([a-f0-9]{16})-(\d{2})$", traceparent)
    if not match:
        return None

    version, trace_id, parent_id, trace_flags = match.groups()

    return SpanContext(
        trace_id=int(trace_id, 16),
        span_id=int(parent_id, 16),
        is_remote=True,
        trace_flags=TraceFlags(int(trace_flags, 16)),
    )
```

### 3. Create Child Spans

```python
from opentelemetry.trace import SpanKind, Status, StatusCode

def call_api(prompt: str, options: dict, promptfoo_context: dict) -> dict:
    traceparent = promptfoo_context.get("traceparent")

    if traceparent:
        span_context = parse_traceparent(traceparent)
        ctx = trace.set_span_in_context(trace.NonRecordingSpan(span_context))

        with tracer.start_as_current_span(
            "my_operation",
            context=ctx,
            kind=SpanKind.SERVER,
        ) as span:
            # Your provider logic here
            result = do_work()
            span.set_status(Status(StatusCode.OK))
            return {"output": result}

    return {"output": do_work()}
```

## Trace-Based Assertions

This example uses several trace assertion types:

```yaml
assert:
  # Count spans matching a pattern
  - type: trace-span-count
    value:
      pattern: 'retrieve_document_*'
      min: 3
      max: 3

  # Check span duration
  - type: trace-span-duration
    value:
      pattern: 'rag_agent_workflow'
      max: 5000 # milliseconds

  # Check for error spans
  - type: trace-error-spans
    value:
      max_count: 0
```

## Viewing Traces

After running an evaluation, view traces in the web UI:

```bash
npx promptfoo@latest view
```

Click on any test result to see the "Trace Timeline" section.

## Dependencies

| Package                                  | Version  | Purpose                       |
| ---------------------------------------- | -------- | ----------------------------- |
| `opentelemetry-api`                      | >=1.44.0 | Core tracing API              |
| `opentelemetry-sdk`                      | >=1.44.0 | SDK implementation            |
| `opentelemetry-exporter-otlp-proto-http` | >=1.44.0 | OTLP HTTP exporter (protobuf) |

## Troubleshooting

### Traces Not Appearing

1. Verify `tracing.enabled: true` in config
2. Check OTLP receiver is running (look for port 4318 in logs)
3. Ensure `processor.force_flush()` is called before returning
4. Check the trace context is properly parsed from `promptfoo_context['traceparent']`

### Import Errors

Install dependencies into the same virtual environment used by Promptfoo.

On macOS/Linux:

```bash
.venv/bin/python -m pip install -r requirements.txt
```

On Windows (PowerShell):

```powershell
.\.venv\Scripts\python.exe -m pip install -r requirements.txt
```

### Connection Refused

Ensure Promptfoo's OTLP receiver is running on port 4318. The receiver starts automatically when `tracing.enabled: true` is set in your config.

## See Also

- [OpenTelemetry Tracing (JavaScript)](../javascript/) - JavaScript version using JSON format
- [Promptfoo Tracing Documentation](https://promptfoo.dev/docs/tracing/)