Opik review: open-source tracing and evals, with a US-hosted cloud

Opik is Comet's open-source platform for tracing, evaluating and tuning LLM applications and agents. The verdict up front: take it if you want traces, LLM-as-a-judge metrics, prompt versions and an optimiser in one Apache 2.0 product, and your team can run its databases. Skip it if you need an EU-hosted cloud or plan to rely on guardrails without a sales conversation.

Opik competes with Langfuse, Arize Phoenix and LangSmith, and the licence is the first thing to check. Opik's repository is Apache 2.0, and its LICENSE file has no carve-out for enterprise directories. Langfuse's core is MIT with the ee/ folders excluded, and Arize Phoenix is under the Elastic License 2.0. The rest of this review covers what you get, what self-hosting takes and where the data goes.

What it is

Opik combines tracing, evaluation and production scoring, with prompt management and an optimiser alongside. The product page calls it 'AI Observability & Evals For the Agentic Era', and the GitHub README says the full platform is free to self-host.

• Apache 2.0 for the whole repository. The LICENSE file is the standard Apache License 2.0 text, copyright Comet ML, Inc.
• Tracing with a decorator. Python installs with pip install opik, and the @opik.track decorator records a function call. The TypeScript SDK imports from opik.
• OpenTelemetry. Any language with an OpenTelemetry SDK can send traces, which Comet describes as first-party support.
• Evaluation. Datasets, experiments, heuristic checks such as Equals, RegexMatch and IsJson, more than 20 LLM-as-a-judge metrics, and a PyTest integration for CI.
• Production and safety. Online evaluation rules score live traces, and guardrails check inputs and outputs. The guardrails are an enterprise feature, as covered below.

How it works

The SDKs batch telemetry and send it over REST to a Java backend that handles the API, authentication and database migrations. A separate Python backend runs evaluators, sandboxed code and optimisation jobs – so in the cloud that code runs on Comet's infrastructure, and self-hosted it runs on yours. Traces and spans go to ClickHouse, projects, datasets and prompts go to MySQL, and attachments go to MinIO. Redis handles caching, rate limiting and queues, ZooKeeper handles cluster coordination, and Nginx serves the React frontend and proxies the API.

Getting started

Install the SDK, set your key and workspace, and decorate the functions you want to trace. Point OPIK_URL_OVERRIDE at a local instance and the same code runs against it.

# pip install opik
# Opik Cloud: set your API key and workspace from the Opik dashboard
# export OPIK_API_KEY="<your-api-key>"
# export OPIK_WORKSPACE="<your-workspace>"
# Self-hosted: point the SDK at your own instance instead
# export OPIK_URL_OVERRIDE="http://localhost:5173/api"

import opik

opik.configure()


@opik.track(name="answer-question")
def answer(question: str) -> str:
    # your model call goes here
    return f"Answer to: {question}"


answer("What does the Apache 2.0 licence allow?")

If you already run OpenTelemetry, skip the SDK and send OTLP traces to Opik. The Cloud endpoint is https://www.comet.com/opik/api/v1/private/otel and a local instance uses http://localhost:5173/api/v1/private/otel. Each request carries the API key in the Authorization header, the project in projectName and the workspace in Comet-Workspace. The page does not say which GenAI semantic conventions Opik maps, so check the attributes you need before you rely on them. For frameworks, the Python docs list integrations for LangChain, LangGraph, LlamaIndex, OpenAI, Anthropic and AWS Bedrock.

Evaluation and judges

Evaluation is where Opik is strongest. The metrics page sorts the library into two groups. The heuristic checks need no model and include Equals, Contains, RegexMatch, IsJson, Levenshtein, ROUGE and BERTScore. The LLM-as-a-judge group has more than 20 metrics, among them Hallucination, Answer Relevance, Context Precision, Moderation, Structured Output Compliance and G-Eval, which takes your own instructions.

Two details matter before you trust a judge score. The metrics page says the Python judge metrics are built on the LiteLLM framework, which is also how the judge model is configured. Every judge call sends the trace text to that model, so the judge provider belongs on your processor list. The page does not say whether scoring runs in the SDK or on the platform, so ask Comet before you point judges at sensitive production traces.

Online evaluation applies the same judges to production. A rule scores a chosen percentage of production traces, and 100 per cent scores them all. Scores are stored as feedback scores on each trace, and an average that crosses a threshold can alert Slack, PagerDuty or a webhook. Each judged trace costs at least one model call, so the sampling rate is also a cost setting, and I would start low. The docs I could open do not say which plans include online evaluation.

Prompts, optimiser and guardrails

The Prompt Library keeps every prompt an agent depends on in one place. Each change creates a new immutable version, numbered v1, v2, v3 and so on. The SDK fetches a prompt with get_prompt or get_chat_prompt from inside a tracked function, and leaving out the version returns the latest. I could not find labels, environments or a rollback command in the docs, so a release process would have to be built on those version numbers.

The optimiser needs the most care. The Opik Agent Optimizer SDK offers six algorithms. MetaPrompt has a reasoning model critique and rewrite the prompt. HRPO batches failures and proposes targeted fixes. Few-Shot Bayesian uses Optuna to choose how many examples to show and in which order. Evolutionary search trades score against prompt length. GEPA runs on Opik datasets and metrics through a wrapper. Parameter leaves the prompt alone and tunes sampling parameters with Bayesian search. Each run calls a model many times, so set a budget and a holdout set before you start one.

Guardrails run inline and synchronously, before a response is returned. The guard types are PII detection, allowed and restricted topics, prompt injection and jailbreak detection, an LLM-as-a-judge check written in plain language, and your own classifier. You group guards into named policies, and the application calls a guardrail by name. The topic and PII checks support English only.

What it costs and where it runs

The open-source edition is free to self-host, and the pricing page lists it with unlimited spans and retention. The cloud plans raise the data question. Here are the plans as the pricing page showed them in October 2026.

Three notes on the table. Extra spans cost $5 per 100,000 on Pro, and extending retention from 60 to 400 days costs $29 per 100,000 spans. The same page says every plan includes unlimited team members, which contradicts the seat caps on the plan cards, so get the seat count in writing before you buy. Researchers, students and educators can use the full Pro plan at no cost.

There are two self-hosting paths. The local guide clones the repository, runs ./opik.sh and opens http://localhost:5173. The docs call it 'perfect to get started but not production-ready', and its --clean option removes all data volumes. For production the docs point to the Kubernetes Helm chart. It bundles ClickHouse and ZooKeeper by default, can use an external ClickHouse from chart version 1.4.2, and configures S3 through S3_BUCKET and S3_REGION, which the page does not describe as bundled. The Kubernetes page does not say how MySQL and Redis are provided, so ask before you size the cluster.

helm repo add opik https://comet-ml.github.io/opik/
helm upgrade --install opik -n opik --create-namespace opik/opik

Here is what the documents say about data. The privacy policy of Comet ML, Inc. says Opik Cloud is hosted in the United States, and that transfers from the EEA and the UK to the US rely on Standard Contractual Clauses. The pricing page lists the data region as US for Free and Pro, and as custom for Enterprise. Neither the privacy policy nor the Trust Center lists a Data Processing Addendum or a subprocessor list, and I found no EU region, so ask for both before you send personal data from the EU. Retention on the self-serve cloud is 60 days by default, which belongs in your record of processing. The wider case for region controls is in GDPR and data residency for LLM APIs.

Opik has two features that touch personal data, and they are easy to confuse. Server-side data anonymisation masks email addresses, phone numbers and your own patterns when people view traces. It is in preview and available on Enterprise only, and it leaves the stored data unchanged. SDK anonymisers work differently. Registered with opik.hooks.add_anonymizer, they detect and replace PII before data is logged, in cloud and self-hosted installations, and the replacement is one-way. The anonymisers page gives examples for email addresses, US phone numbers, Social Security numbers and credit card numbers. Names and addresses need your own rules.

Where it falls short

• No EU cloud region that I could find. Opik Cloud is US-hosted on Free and Pro, and Enterprise lists a custom region. For EU personal data, that points to self-hosting or an Enterprise contract.
• Guardrails and server-side masking sit behind sales. Both are enterprise features in the docs, and the guardrails server runs its own models, so budget for the compute as well as the contract.
• The self-hosted stack is heavy. A production install runs ClickHouse, MySQL, Redis, MinIO, ZooKeeper and both backends. The local guide says it is not meant for production, so plan a platform project, not a weekend.
• Documentation gaps. The pricing page contradicts itself on seats, the metrics page does not say where scoring runs, the Prompt Library shows no labels or rollback, and the OpenTelemetry page does not name the GenAI conventions it maps.
• Versions move quickly. The PyPI package was at 2.2.96 when I checked, and the chart and the SDK should match, so pin both.

Verdict

Opik is a strong Apache 2.0 option for tracing, judges, prompt versions and optimisation in one product. Its gaps are not in the feature list. They are the enterprise gate on guardrails, the US-hosted cloud, and the running cost of the self-hosted stack. Take it for a platform team that already runs Kubernetes and ClickHouse and needs the whole repository under Apache 2.0. Look elsewhere when the decision turns on EU hosting or a simpler install.

1. Adopt it if one team owns traces, evaluation and prompts, and you want the full repository under Apache 2.0.
2. Adopt it if you can run ClickHouse, MySQL and MinIO with the same care as your other data stores.
3. Do not adopt it if you need EU-hosted cloud processing or a DPA you can read before signing, because I could not find either.
4. Do not adopt it if guardrails are a must-have and you cannot take an enterprise contract.

Pick Langfuse if you want an MIT core with the ee/ exception and a platform you can compare on your own traces. Pick Arize Phoenix if you want a self-hosted tool and can accept the Elastic License 2.0 that governs it. Pick LangSmith if you want a hosted product with cloud data in the US or the EU, and you accept per-seat pricing, which starts at $39 a month per seat on Plus, with self-hosting only on Enterprise.

Sources

• Opik repository README on GitHub
• Opik LICENSE file: Apache License 2.0
• Opik on PyPI: package metadata
• Opik product page
• Opik pricing
• Opik docs: run locally with Docker Compose
• Opik docs: Kubernetes deployment with Helm
• Opik docs: platform architecture
• Opik docs: tracing getting started
• Opik docs: OpenTelemetry integration
• Opik docs: evaluation metrics overview
• Opik docs: online evaluation rules
• Opik docs: Prompt Library overview
• Opik docs: optimisation algorithms
• Opik docs: guardrails overview
• Opik docs: guardrails server
• Opik docs: SDK anonymizers
• Opik docs: data anonymization
• Comet ML privacy policy
• Comet Trust Center
• Langfuse repository README (licence)
• Arize Phoenix repository README (licence)
• LangChain pricing (LangSmith plans)