Blog/Topic
LLMOps and evals: model routing and cost, prompt caching, eval gates in CI, and tracing for LLM agents in production.
LLMOps & evals
·11 min read
Claude plan prices as of October 2026, how the five-hour and weekly usage limits work, Team seats, API token prices and how to pick a plan.
LLMOps & evals
·15 min read
Claude Haiku 5.5 pricing by tier, benchmarks against Haiku 4.5 and Sonnet 5.5, effort levels, and which Sonnet 5.5 jobs can move to it.
LLMOps & evals
·15 min read
DeepSeek V4.1 Flash 1017 deep dive: the encoder-decoder split, CSA2 cache sharing, FP4 KV cache, benchmarks, and the real cost per agent task.
LLMOps & evals
·10 min read
How LLM rate limits work, which errors to retry, backoff with jitter, retrying in one layer only, and fallback chains that do not break your output.
LLMOps & evals
·8 min read
Prompt versioning in production: what belongs in a version, git vs a prompt registry, labels and rollbacks, eval gates in CI, and cache side effects.
LLMOps & evals
·9 min read
When to use the Anthropic, OpenAI and Gemini batch APIs, how limits, result matching and failures work, and how batch discounts stack with prompt caching.
LLMOps & evals
·11 min read
When a small model beats a frontier API: cost per resolved case, latency, privacy, distillation, quantization, routing and an eval plan for your traffic.
LLMOps & evals
·10 min read
Reasoning effort on Claude, OpenAI and Gemini: adaptive thinking, what thinking tokens cost, and how to pick a level with an effort sweep.
LLMOps & evals
·9 min read
How structured outputs and constrained decoding work on OpenAI, Claude, Gemini and vLLM, which schema rules matter, and what you still validate.
LLMOps & evals
·9 min read
How to evaluate AI agents: outcome vs trajectory graders, trajectory match modes, pass^k, isolated environments and LLM judges that hold up.
LLMOps & evals
·10 min read
A playbook for switching LLM models: deprecation timelines, contract fixes, a regression suite, prompt re-tuning, shadow traffic and canary rollout.
LLMOps & evals
·6 min read
How semantic caches reuse LLM answers for similar questions, how they differ from prompt caching, and how to avoid wrong hits, leaks and stale answers.
LLMOps & evals
·10 min read
Self-hosting an LLM for GDPR: when it is required, GPU memory for open-weight models, EU prices as of October 2026 and break-even per million tokens.
LLMOps & evals
·8 min read
DeepEval: LLM checks as pytest-style tests with built-in judge metrics. The library is free and Apache-2.0; judge calls and data flow are the real cost.
LLMOps & evals
·8 min read
DSPy compiles prompts from signatures, a metric and examples: what its optimisers cost in model calls, and when a hand-written prompt still wins.
LLMOps & evals
·8 min read
llama.cpp runs open models in plain C and C++ on Metal, CUDA, Vulkan or the CPU. I cover GGUF quants, llama-server and where it falls short.
LLMOps & evals
·7 min read
Opik: traces, LLM-as-a-judge metrics, prompt versions and an optimiser on one Apache 2.0 platform. Free to self-host, Pro cloud $19/month, US-hosted.
LLMOps & evals
·11 min read
Three open TTS models on one laptop: 96 articles became 1,512 minutes of narration, via chunked synthesis and AAC at 192 kbit/s.
LLMOps & evals
·11 min read
Ollama serves open models over one HTTP API on your own hardware. What it does well, where throughput falls short, and what the MIT licence does not cover.
LLMOps & evals
·11 min read
Portkey: an OpenAI-compatible gateway for retries, fallbacks, caching, guardrails and cost tracking. Its config, latency cost and when to self-host.
LLMOps & evals
·8 min read
Claude Opus 5.5 is #1 of 211 models on Artificial Analysis with 58 points. At medium effort it matches Opus 5 for $1.34 per task instead of $5.86.
LLMOps & evals
·10 min read
LLM routing with typed decisions: Choice, Score and yes-probability answers with calibrated confidence, thresholds and human hand-off, using Jev.
LLMOps & evals
·10 min read
Langfuse: LLM traces, prompt versions and experiments on one MIT-licensed platform. What self-hosting costs, how unit pricing adds up, and where it loses.
LLMOps & evals
·12 min read
How to trace LLM agents with OpenTelemetry: GenAI semantic conventions and their status, span tree, token metrics, sampling, PII, evals and tool options.
LLMOps & evals
·10 min read
OpenRouter: 500+ models from 80+ providers behind one OpenAI-compatible endpoint, with fallbacks and pass-through pricing. Costs and where it breaks.
LLMOps & evals
·10 min read
Braintrust: traces become datasets and gated experiments. Starter and Pro tiers, open-source parts, and where Phoenix, Langfuse or LangSmith win.
LLMOps & evals
·10 min read
Promptfoo in 2026: MIT-licensed evals and red teaming, now inside OpenAI. What it does well, where the YAML approach breaks, and what the tiers cost.
LLMOps & evals
·10 min read
Arize Phoenix: an ELv2-licensed tracing and eval server on your own database. Strengths, operating cost, and where Langfuse, Braintrust or LangSmith win.
LLMOps & evals
·10 min read
Helicone: an Apache-2.0 LLM gateway and observability platform. What its proxy architecture buys, what it costs you, and how self-hosting really looks.
LLMOps & evals
·11 min read
vLLM turns a Hugging Face checkpoint into an OpenAI-compatible server. What PagedAttention and continuous batching buy, and what running it actually costs.
LLMOps & evals
·9 min read
Prompt caching, cheap-model-first routing and batch APIs are the levers that cut LLM cost and latency in production. Here is how to use each one.
LLMOps & evals
·10 min read
LangSmith review: per-trace billing, 14 and 180 day retention, OpenTelemetry ingestion, offline and online evals, and its pull toward an agent runtime.
LLMOps & evals
·10 min read
LiteLLM: the MIT-licensed OpenAI-compatible gateway most platform teams run in front of providers. What it does well, what it costs, where it breaks.
LLMOps & evals
·10 min read
Ragas scores RAG and agent pipelines on faithfulness, context precision and recall, generates test data, and tracks runs. Apache-2.0; judge billed apart.
LLMOps & evals
·8 min read
LLM evals turn a vibe check into a test suite: error analysis on real traces, grader choice, a validated LLM judge, pass^k and a CI gate.
LLMOps & evals
·12 min read
Fine-tuning vs RAG vs prompting: what each changes, what it costs, who still offers SFT, DPO and RFT in 2026, and a decision tree.