LLMOps & evals
·7 min read
Opik review: open-source tracing and evals, with a US-hosted cloud
Opik: traces, LLM-as-a-judge metrics, prompt versions and an optimiser on one Apache 2.0 platform. Free to self-host, Pro cloud $19/month, US-hosted.
Blog/Tag
Articles about Tracing
7 articles tagged Tracing, newest first. Each one is sourced and written for people who ship.
AI agents
·10 min read
OpenAI Agents SDK: a small agent runtime with sharp edges
A review of the OpenAI Agents SDK: the runner loop, tracing, guardrails and approvals, plus what the release churn and the Responses-only features cost.
- Agent runtime
- Tracing
- Guardrails
LLMOps & evals
·10 min read
Langfuse review: tracing, prompts and evals you can host yourself
Langfuse: LLM traces, prompt versions and experiments on one MIT-licensed platform. What self-hosting costs, how unit pricing adds up, and where it loses.
LLMOps & evals
·12 min read
Observability for LLM agents with OpenTelemetry: traces, tokens, PII and evals
How to trace LLM agents with OpenTelemetry: GenAI semantic conventions and their status, span tree, token metrics, sampling, PII, evals and tool options.
LLMOps & evals
·10 min read
Braintrust review: eval-first observability with a hard meter
Braintrust: traces become datasets and gated experiments. Starter and Pro tiers, open-source parts, and where Phoenix, Langfuse or LangSmith win.
- Evaluations
- Observability
- LLM-as-a-judge
LLMOps & evals
·10 min read
Arize Phoenix review: LLM tracing and evals you host yourself
Arize Phoenix: an ELv2-licensed tracing and eval server on your own database. Strengths, operating cost, and where Langfuse, Braintrust or LangSmith win.
LLMOps & evals
·10 min read
LangSmith: an observability product that grew into an agent platform
LangSmith review: per-trace billing, 14 and 180 day retention, OpenTelemetry ingestion, offline and online evals, and its pull toward an agent runtime.
- Tracing
- LLM evals
- OpenTelemetry