[{"data":1,"prerenderedAt":615},["ShallowReactive",2],{"blog-braintrust-en":3},{"slug":4,"published":5,"minutes":6,"category":7,"tags":8,"keywords":14,"about":21,"sources":31,"cover":56,"og":57,"expertise":58,"locales":59,"lang":60,"title":63,"description":64,"coverAlt":65,"metaTitle":66,"takeaways":67,"faq":73,"toc":86,"blocks":111,"others":376},"braintrust","2026-07-07",10,"llmops",[9,10,11,12,13],"Evaluations","Observability","LLM-as-a-judge","CI gates","Tracing",[4,15,16,17,18,19,20],"braintrust pricing","braintrust eval","autoevals library","braintrust vs langfuse","llm evaluation platform","eval driven development",[22,25,28],{"name":23,"url":24},"Braintrust","https:\u002F\u002Fwww.braintrust.dev",{"name":26,"url":27},"Continuous integration","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FContinuous_integration",{"name":29,"url":30},"Observability (software)","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FObservability_(software)",[32,35,38,41,44,47,50,53],{"title":33,"url":34},"Braintrust pricing: plans, credits and usage rates","https:\u002F\u002Fwww.braintrust.dev\u002Fpricing",{"title":36,"url":37},"Braintrust documentation: plans and limits","https:\u002F\u002Fwww.braintrust.dev\u002Fdocs\u002Fplans-and-limits",{"title":39,"url":40},"Braintrust documentation: evaluation quickstart","https:\u002F\u002Fwww.braintrust.dev\u002Fdocs\u002Fevaluation-quickstart",{"title":42,"url":43},"Braintrust documentation: get started","https:\u002F\u002Fwww.braintrust.dev\u002Fdocs",{"title":45,"url":46},"GitHub: Braintrust organisation repositories","https:\u002F\u002Fgithub.com\u002Forgs\u002Fbraintrustdata\u002Frepositories",{"title":48,"url":49},"Arize Phoenix documentation: self-hosting","https:\u002F\u002Farize.com\u002Fdocs\u002Fphoenix\u002Fself-hosting",{"title":51,"url":52},"Langfuse pricing: cloud plans and billable units","https:\u002F\u002Flangfuse.com\u002Fpricing",{"title":54,"url":55},"LangChain pricing: LangSmith plans","https:\u002F\u002Fwww.langchain.com\u002Fpricing","\u002Fimages\u002Fblog\u002Fbraintrust\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fbraintrust\u002Fog.jpg","ai-engineer",[60,61,62],"en","de","hu","Braintrust review: eval-first observability with a hard meter","Braintrust turns production traces into datasets and gated experiments. What Starter and Pro really include, which parts are open source, and where Phoenix, Langfuse and LangSmith win.","Loose squares gathered by a wide arrow into a row, then one arrow crossing a gate.","Braintrust: eval-first observability · Balázs Csorba",[68,69,70,71,72],"Starter is genuinely usable but small: 1 GB of processed data, 10,000 scores and 14 days of retention, then Pro at $249 a month for 5 GB, 50,000 scores and 30 days.","Processed data is measured at ingestion, so deleting traces does not reduce the bill, and overage runs $4 per GB on Starter and $3 per GB on Pro.","The platform is closed source while the tooling is open: SDKs for six languages under Apache-2.0, autoevals under MIT and the bt CLI under Apache-2.0.","BYOC and self-hosted deployment require Enterprise, and SAML SSO, audit logging, custom retention and SOC 2 attestation are Enterprise rows as well.","There is no tier between $0 and $249, which makes Braintrust cheap for a solo evaluation project and expensive for a team that only wanted dashboards.",[74,77,80,83],{"q":75,"a":76},"How much does Braintrust cost?","Starter is $0 with $10 of model credits, 1 GB of processed data, 10,000 scores and 14 days of retention, plus unlimited users, projects, datasets, playgrounds and experiments. Pro is $249 a month for $100 of credits, 5 GB, 50,000 scores, 30 days of retention, custom charts, environments, RBAC and priority support. Enterprise is custom-priced and adds BYOC, self-hosted deployment, SAML or OIDC single sign-on, audit logging, a BAA and an uptime SLA.",{"q":78,"a":79},"Is Braintrust open source?","The platform is not; much of the tooling is. The Python, TypeScript, Go, Ruby, Java and C# SDKs are Apache-2.0, autoevals is MIT with about a thousand GitHub stars, braintrust-proxy is MIT and the agentbehavior standard is Apache-2.0. There is no self-hosted edition of the service below the Enterprise plan.",{"q":81,"a":82},"What counts as a score in the pricing?","A score is one recorded evaluation result, whether it came from an LLM-as-a-judge call, an autoevals scorer or custom code, attached to a trace or an experiment row. Starter includes 10,000 a month and then charges $2.50 per thousand; Pro includes 50,000 and then $1.50 per thousand. Judge-based scoring multiplies quickly across a dataset, so the score quota is often reached before the data quota.",{"q":84,"a":85},"Braintrust or Arize Phoenix?","Choose Braintrust when evaluation belongs in CI and a platform fee is acceptable: the Eval API, the scorer library and the eval-action workflow are one integration. Choose Phoenix when traces cannot leave your infrastructure, because Braintrust's BYOC and self-hosted deployments require an Enterprise contract while Phoenix is free to self-host under the Elastic License 2.0.",[87,90,93,96,99,102,105,108],{"id":88,"title":89},"what-it-is","What it is",{"id":91,"title":92},"how-it-works","How it works",{"id":94,"title":95},"getting-started","Getting started",{"id":97,"title":98},"pricing","What it costs",{"id":100,"title":101},"open-or-closed","What is open and what is not",{"id":103,"title":104},"where-it-shingles","Where it falls short",{"id":106,"title":107},"verdict","Verdict",{"id":109,"title":110},"sources","Sources",[112,121,124,127,130,148,151,152,159,165,168,169,176,178,185,192,193,196,241,244,250,251,254,266,269,270,273,319,322,327,328,331,343,348,349],{"type":113,"content":114},"paragraph",[115,116,120],"Braintrust is a hosted platform for instrumenting an LLM application, keeping its traces and scoring them against datasets in a way that repeats. Its centre of gravity is the evaluation: an ",{"tag":117,"children":118},"code",[119],"Eval()"," call that runs a task over a dataset, scores the outputs and stores every run as a comparable experiment. The position of this review is that Braintrust is the strongest eval-first product in its class and the most tightly metered of its peers — the engineering is excellent, and the billing rewards teams who know exactly how much data they intend to send.",{"type":113,"content":122},[123],"It covers three jobs that are usually split between tools: tracing in production, offline evaluation, and the gate between them in continuous integration. Arize Phoenix and Langfuse compete on the open-source side, LangSmith for shops already building on LangGraph. What it tends to replace is worse: a set of test cases that call the model, print scores to stdout and get deleted the week they turn flaky.",{"type":125,"level":126,"id":88,"text":89},"heading",2,{"type":113,"content":128},[129],"Braintrust is a SaaS product from Braintrust Data, built on three objects: logs, which are traces from the application; datasets, which are rows of input and expected output; and experiments, which run a task over a dataset with scorers attached. The documentation describes a five-step workflow — instrument, observe, annotate, evaluate, deploy — and ships SDKs for Python, TypeScript, Go, Ruby, Java and C#. The platform itself is closed source; much of the tooling around it is not.",{"type":131,"ordered":132,"items":133},"list",false,[134,136,138,140,142,144,146],[135],"Licence: platform closed source; SDKs Apache-2.0, autoevals MIT, braintrust-proxy MIT",[137],"Instrumentation: braintrust.auto_instrument() wraps the provider clients already in the process, plus a bt CLI for setup from a coding agent",[139],"Evaluation: Eval() over a dataset with autoevals scorers, custom code or LLM-as-a-judge",[141],"Plans: Starter at $0, Pro at $249 a month and Enterprise by quotation, all with unlimited users, projects, datasets and experiments",[143],"Metering: processed data in gigabytes, scores in thousands, model credits in dollars",[145],"Deployment: SaaS on Starter and Pro, with BYOC and self-hosted deployment only on Enterprise",[147],"Extras: playgrounds, environments and custom charts on Pro and above, and Loop, a built-in agent that writes scorers, test cases and prompt iterations",{"type":113,"content":149},[150],"That last row is the tell. Braintrust assumes the platform is someone else's problem and the evaluation is yours, and teams that accept the premise get a very short loop from a production failure to a dataset row to a gated release. Teams that need the whole system inside their own account find that the price of that loop is an Enterprise contract, not the $249 tier.",{"type":125,"level":126,"id":91,"text":92},{"type":113,"content":153},[154,155,158],"Instrumentation happens in the process: ",{"tag":117,"children":156},[157],"braintrust.auto_instrument()"," patches the provider client the application already uses, so spans arrive with token costs attached and with no sidecar to deploy. Everything lands in a project, and logs, datasets and experiments share that project, which is what makes a production trace promotable into a test case.",{"type":160,"src":161,"alt":162,"wide":163,"caption":164},"image","\u002Fimages\u002Fblog\u002Fbraintrust\u002Fdiagram-8.webp","The Braintrust loop from trace to gated release An instrumented application logs traces and their costs to a project. Production traces become rows of a dataset, an experiment runs a task over that dataset with scorers attached, and a comparison of two runs gates the pull request before the change goes back to the application.",true,"Traces, datasets and experiments share one project, so a production failure can become a test case without an export.",{"type":113,"content":166},[167],"The distinctive part is comparison. An experiment is a permanent record, and a second run under a new experiment name is diffed against the first in the interface. The documentation's own quickstart leans on this: the first run scores near zero under ExactMatch because the model answers with a sentence, the prompt is tightened to return a bare title, and the second experiment reports the gain as a score delta rather than an impression.",{"type":125,"level":126,"id":94,"text":95},{"type":113,"content":170},[171,172,175],"Two keys and a file: ",{"tag":117,"children":173},[174],"BRAINTRUST_API_KEY"," for the platform, your provider key for the model, and one Python file holding data, task and scorer. The snippet below is the documented quickstart shape with the repetition trimmed out:",{"type":117,"code":177},"# export BRAINTRUST_API_KEY=...  and  export OPENAI_API_KEY=...\n# pip install braintrust openai autoevals\nimport braintrust\nfrom braintrust import Eval\nfrom autoevals import ExactMatch\nfrom openai import OpenAI\n\nbraintrust.auto_instrument()  # traces every call this process makes\nclient = OpenAI()\n\ndef task(input):\n    r = client.responses.create(\n        model=\"gpt-5-mini\",\n        input=[{\"role\": \"system\", \"content\": \"Name the film.\"},\n               {\"role\": \"user\", \"content\": input}],\n    )\n    return r.output_text\n\nEval(\n    \"movie-matcher\",\n    experiment_name=\"baseline-v1\",\n    data=[{\"input\": \"A detective hunts a killer through the seven deadly sins.\",\n           \"expected\": \"Se7en\"}],\n    task=task,\n    scores=[ExactMatch],\n)",{"type":113,"content":179},[180,181,184],"Run it with ",{"tag":117,"children":182},[183],"bt eval movie_matcher.py"," and the terminal prints a link to the experiment; plain Python works as well, because the CLI only wraps the entry point. That same file is what a pull request runs: the eval-action GitHub workflow executes it against main and fails the build when a score regresses.",{"type":186,"variant":187,"title":188,"body":189},"callout","warn","Scoring is billed, not just computed",[190],[191],"Every scorer call that records a score counts against the monthly quota — 10,000 on Starter, 50,000 on Pro — and judge-based scoring is included, at one model request per row. Scorers should target the failures that matter, because a full run across a large dataset can cost more than the trace that produced it.",{"type":125,"level":126,"id":97,"text":98},{"type":113,"content":194},[195],"Three plans and no middle step: the pricing page lists Starter at $0, Pro at $249 a month and Enterprise by quotation, with users, projects, datasets, playgrounds and experiments unlimited on all three. What differs is the meter:",{"type":197,"head":198,"rows":209},"table",[199,201,203,205,207],[200],"Plan",[202],"Price",[204],"Processed data",[206],"Scores",[208],"Retention",[210,221,232],[211,213,215,217,219],[212],"Starter",[214],"$0",[216],"1 GB, then $4\u002FGB",[218],"10,000, then $2.50 per 1,000",[220],"14 days",[222,224,226,228,230],[223],"Pro",[225],"$249 a month",[227],"5 GB, then $3\u002FGB",[229],"50,000, then $1.50 per 1,000",[231],"30 days, then $0.50 per GB-month",[233,235,237,238,239],[234],"Enterprise",[236],"Custom",[236],[236],[240],"Custom, up to 365 days",{"type":113,"content":242},[243],"Model credits run alongside: $10 a month on Starter and $100 on Pro cover built-in models and platform features such as Topics, after which token rates apply. The tier jump is binary — a team that outgrows 1 GB and 14 days pays the full $249, though the docs offer six to twelve months of Pro to qualifying new startup customers.",{"type":186,"variant":245,"title":246,"body":247},"note","Processed data is counted at ingestion",[248],[249],"The plans page is explicit that processed data measures bytes at ingestion and that deleting data does not reduce it, so pruning noisy traces never shrinks a bill retroactively. Experiment results are kept for up to 365 days on every plan, while logs and playground data follow the 14-day or 30-day window, which is also the horizon of what the lower tiers can still query.",{"type":125,"level":126,"id":100,"text":101},{"type":113,"content":252},[253],"The split deserves its own section, because the sentence Braintrust is open source is wrong in the way that matters: below Enterprise you cannot run the platform yourself. What is open is the instrumentation and the scoring tooling around it.",{"type":131,"ordered":132,"items":255},[256,258,260,262,264],[257],"braintrust-sdk-python, -javascript, -go and -ruby: Apache-2.0",[259],"autoevals: MIT, about a thousand stars, the scorer library the quickstart imports",[261],"braintrust-proxy: MIT, a proxy in front of model traffic",[263],"agentbehavior: Apache-2.0, an open standard and dataset for agent behaviour",[265],"The bt CLI and the coding-agent plugins: Apache-2.0 or MIT; the hosted web application is not published",{"type":113,"content":267},[268],"For an engineering team that distinction is mostly philosophical until procurement asks, and then it is the whole discussion. It also sets the exit path: the SDKs can be repointed at another backend without rewriting the application, but datasets, experiments and dashboards live in Braintrust's service, and scheduled export to S3 or Google Cloud Storage is an Enterprise feature.",{"type":125,"level":126,"id":103,"text":104},{"type":113,"content":271},[272],"The first weakness is arithmetic. There is no tier between $0 and $249, the free tier's 1 GB and 14 days are a demo budget for a production agent, and scoring meters on top of ingestion, so a team running a judge across every production trace hits the score quota before the data quota. The second is what the lower tiers do not carry: SAML single sign-on, audit logging, custom retention, export automations, SOC 2 attestation and a business associate agreement are Enterprise rows, while Starter is limited to the owner permission group and one human-review score per project. The third is contractual drift: the documentation still carries a legacy-plan note for accounts created before 16 March 2026, so an evaluation done under the old terms is worth redoing.",{"type":197,"head":274,"rows":283},[275,277,279,281],[276],"Tool",[278],"Free tier",[280],"Paid entry",[282],"Self-host",[284,292,301,310],[285,286,288,290],[23],[287],"1 GB, 10,000 scores, 14 days",[289],"Pro $249 a month",[291],"Enterprise only",[293,295,297,299],[294],"Arize Phoenix",[296],"No caps on a local install",[298],"AX Pro $50 a month",[300],"Free under ELv2",[302,304,306,308],[303],"Langfuse",[305],"50,000 units, 30 days, 2 users",[307],"Core $29 a month",[309],"Free, Docker Compose",[311,313,315,317],[312],"LangSmith",[314],"1 seat, 5,000 base traces",[316],"Plus $39 a seat",[318],"Enterprise add-on",{"type":113,"content":320},[321],"Read honestly, Braintrust and Phoenix are not competing on the same axis: one is priced for teams buying an evaluation process, the other for teams owning the infrastructure. Langfuse is the closest direct substitute at roughly a third of the entry price, and LangSmith is the natural pick when the application already sits on LangGraph and first-party trace formats matter more than the scorer library.",{"type":323,"content":324},"quote",[325,326],"SaaS is available on all plans. BYOC and self-hosted deployments, which keep data in your own cloud, require Enterprise."," — Braintrust documentation, plans and limits",{"type":125,"level":126,"id":106,"text":107},{"type":113,"content":329},[330],"Braintrust earns its price when evaluation is a recurring engineering ritual rather than a one-off script: the experiment model, the scorer library and the CI action form a loop that is tedious to assemble from parts. It is a poor fit for a team whose main need is trace storage behind a dashboard, because that is a $249 habit with a data bill attached to it.",{"type":131,"ordered":163,"items":332},[333,335,337,339,341],[334],"Choose it when a pull request should fail on an eval regression and you want the scorer library to already exist.",[336],"Choose it when several roles — engineers, reviewers, a product manager — need the same experiments and dashboards; seats are unlimited on every plan.",[338],"Choose it when volume is predictable: 5 GB and 50,000 scores a month is a known quantity, and the overage rates are published.",[340],"Do not choose it when traces must stay inside your own account; BYOC and self-hosting need an Enterprise contract.",[342],"Do not choose it when the workload is chatty and unbounded: metering at ingestion plus per-score charges makes a noisy agent expensive in a way per-seat tools are not.",{"type":186,"variant":245,"title":344,"body":345},"Bottom line",[346],[347],"For the price of one engineer's part-time attention a month, Braintrust supplies the evaluation discipline most teams intend to build and never finish. Buy it for the evaluation loop, not for trace storage; for storage alone, self-hosted Phoenix or a $29 Langfuse plan does the same job at a fraction of the cost.",{"type":125,"level":126,"id":109,"text":110},{"type":131,"ordered":163,"items":350},[351,355,358,361,364,367,370,373],[352],{"tag":353,"href":34,"children":354},"a",[33],[356],{"tag":353,"href":37,"children":357},[36],[359],{"tag":353,"href":40,"children":360},[39],[362],{"tag":353,"href":43,"children":363},[42],[365],{"tag":353,"href":46,"children":366},[45],[368],{"tag":353,"href":49,"children":369},[48],[371],{"tag":353,"href":52,"children":372},[51],[374],{"tag":353,"href":55,"children":375},[54],[377,434,501,559],{"slug":378,"published":379,"minutes":380,"category":7,"tags":381,"keywords":387,"about":401,"sources":409,"cover":428,"og":429,"expertise":58,"locales":430,"lang":60,"title":431,"description":432,"coverAlt":433},"deepseek-v4-1-flash-1017","2026-10-17",15,[382,383,384,385,386],"DeepSeek","Open-weight models","KV cache","Mixture of experts","LLM cost",[388,389,390,391,392,393,394,395,396,397,398,399,400],"DeepSeek V4.1 Flash 1017","DeepSeek V4.1 Flash","Causal Encoder-Decoder","Compressed Sparse Attention 2","CSA2","FP4 KV cache","KV cache compression","SWA Bounded Replay","Engram conditional memory","DSpark speculative decoding","DeepSeek pricing","DeepSWE benchmark","open-weight coding model",[402,404,406],{"name":382,"url":403},"https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FDeepSeek",{"name":385,"url":405},"https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FMixture_of_experts",{"name":407,"url":408},"Transformer (deep learning architecture)","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FTransformer_(deep_learning_architecture)",[410,413,416,419,422,425],{"title":411,"url":412},"DeepSeek-AI: DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression (arXiv:2609.19969)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.19969",{"title":414,"url":415},"Hugging Face: deepseek-ai\u002FDeepSeek-V4.1-Flash-1017 model card","https:\u002F\u002Fhuggingface.co\u002Fdeepseek-ai\u002FDeepSeek-V4.1-Flash-1017",{"title":417,"url":418},"OpenRouter: DeepSeek V4.1 Flash 1017, pricing and providers","https:\u002F\u002Fopenrouter.ai\u002Fdeepseek\u002Fdeepseek-v4.1-flash-1017",{"title":420,"url":421},"The Decoder: Deepseek V4.1 Flash 1017 beats larger rivals in agentic tasks while once again massively cutting per-task costs","https:\u002F\u002Fthe-decoder.com\u002Fdeepseek-v4-1-flash-1017-beats-larger-rivals-in-agentic-tasks-while-once-again-massively-cutting-per-task-costs\u002F",{"title":423,"url":424},"AI Weekly: DeepSeek opens V4.1-Flash-1017 weights and $0.75 Search API","https:\u002F\u002Faiweekly.co\u002Falerts\u002Fdeepseek-opens-v41-flash-1017-weights-and-075-search-api",{"title":426,"url":427},"DeepSeek on X: DeepSeek-V4.1-Flash-1017 announcement","https:\u002F\u002Fx.com\u002Fdeepseek_ai\u002Fstatus\u002F2111375640717033777","\u002Fimages\u002Fblog\u002Fdeepseek-v4-1-flash-1017\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fdeepseek-v4-1-flash-1017\u002Fog.jpg",[60,61,62],"DeepSeek V4.1 Flash 1017: 890 bytes of KV cache per token, and what that buys an agent","A technical deep dive into DeepSeek V4.1 Flash 1017: the encoder-decoder split, CSA2 cache sharing, FP4 KV cache, the benchmarks and the real cost per agent task.","A tall stack of thin cards on the left, an arrow, and the same cards pressed into a stack a quarter of the height on the right.",{"slug":435,"published":379,"minutes":6,"category":7,"tags":436,"keywords":442,"about":454,"sources":464,"cover":495,"og":496,"expertise":58,"locales":497,"lang":60,"title":498,"description":499,"coverAlt":500},"llm-rate-limits-retries-provider-fallback",[437,438,439,440,441],"Rate limits","Retries","Fallback","Reliability","LLM gateway",[443,444,445,446,447,448,449,450,451,452,453],"llm rate limit handling","429 too many requests llm api","exponential backoff with jitter","anthropic 529 overloaded error","openai rate limit retry","llm provider fallback","litellm fallbacks","retry-after header","llm circuit breaker","multi provider llm failover","claude api rate limits",[455,458,461],{"name":456,"url":457},"Token bucket","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FToken_bucket",{"name":459,"url":460},"Exponential backoff","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FExponential_backoff",{"name":462,"url":463},"Circuit breaker design pattern","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FCircuit_breaker_design_pattern",[465,468,471,474,477,480,483,486,489,492],{"title":466,"url":467},"Claude Platform Docs: Rate limits","https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fapi\u002Frate-limits",{"title":469,"url":470},"Claude Platform Docs: API errors","https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fapi\u002Ferrors",{"title":472,"url":473},"OpenAI API Docs: Rate limits","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fguides\u002Frate-limits",{"title":475,"url":476},"AWS Builders Library: Timeouts, retries and backoff with jitter","https:\u002F\u002Fbuilder.aws.com\u002Fcontent\u002F3EumjoZascWd1oZiEgL8ORlv3qE\u002Ftimeouts-retries-and-backoff-with-jitter",{"title":478,"url":479},"MDN: Retry-After header","https:\u002F\u002Fdeveloper.mozilla.org\u002Fen-US\u002Fdocs\u002FWeb\u002FHTTP\u002FReference\u002FHeaders\u002FRetry-After",{"title":481,"url":482},"Martin Fowler: Circuit Breaker (2014)","https:\u002F\u002Fmartinfowler.com\u002Fbliki\u002FCircuitBreaker.html",{"title":484,"url":485},"LiteLLM Docs: Router, retries and cooldowns","https:\u002F\u002Fdocs.litellm.ai\u002Fdocs\u002Frouting",{"title":487,"url":488},"LiteLLM Docs: Proxy fallbacks and reliability","https:\u002F\u002Fdocs.litellm.ai\u002Fdocs\u002Fproxy\u002Freliability",{"title":490,"url":491},"OpenRouter Docs: Model fallbacks","https:\u002F\u002Fopenrouter.ai\u002Fdocs\u002Fguides\u002Frouting\u002Fmodel-fallbacks",{"title":493,"url":494},"AI SDK Docs: Settings (maxRetries, timeout)","https:\u002F\u002Fai-sdk.dev\u002Fdocs\u002Fai-sdk-core\u002Fsettings","\u002Fimages\u002Fblog\u002Fllm-rate-limits-retries-provider-fallback\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fllm-rate-limits-retries-provider-fallback\u002Fog.jpg",[60,61,62],"LLM rate limits in production: retries, backoff and provider fallback","How LLM rate limits work, which errors to retry, backoff with jitter, retrying in one layer only, and fallback chains that do not break your output.","Small squares climbing three steps up to a wide rectangle, two more rectangles behind it in a row.",{"slug":502,"published":379,"minutes":503,"category":7,"tags":504,"keywords":509,"about":520,"sources":528,"cover":553,"og":554,"expertise":58,"locales":555,"lang":60,"title":556,"description":557,"coverAlt":558},"prompt-versioning-management",8,[505,506,507,508,303],"Prompt management","Versioning","LLMOps","Evals",[510,511,512,513,514,515,516,517,518,519],"prompt versioning","prompt management","prompt registry","version control for prompts","langfuse prompt management","mlflow prompt registry","prompts in git","openai prompt object deprecation","prompt rollback production","prompt ci evaluation",[521,524,527],{"name":522,"url":523},"Prompt engineering","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FPrompt_engineering",{"name":525,"url":526},"Version control","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FVersion_control",{"name":26,"url":27},[529,532,535,538,541,544,547,550],{"title":530,"url":531},"OpenAI API Docs: Migrate from prompt objects","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fguides\u002Fprompting\u002Fmigrate-from-prompt-object",{"title":533,"url":534},"OpenAI API Docs: Prompt engineering (reusable prompts deprecation note)","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fguides\u002Fprompt-engineering",{"title":536,"url":537},"Langfuse Docs: Prompt version control","https:\u002F\u002Flangfuse.com\u002Fdocs\u002Fprompt-management\u002Ffeatures\u002Fprompt-version-control",{"title":539,"url":540},"Langfuse Docs: Prompt caching in the SDKs","https:\u002F\u002Flangfuse.com\u002Fdocs\u002Fprompt-management\u002Ffeatures\u002Fcaching",{"title":542,"url":543},"Langfuse Docs: Link prompts to traces","https:\u002F\u002Flangfuse.com\u002Fdocs\u002Fprompt-management\u002Ffeatures\u002Flink-to-traces",{"title":545,"url":546},"MLflow Docs: Prompt Registry","https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fgenai\u002Fprompt-registry\u002F",{"title":548,"url":549},"Promptfoo Docs: GitHub Action","https:\u002F\u002Fwww.promptfoo.dev\u002Fdocs\u002Fintegrations\u002Fgithub-action\u002F",{"title":551,"url":552},"Claude Platform Docs: Prompt caching","https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fbuild-with-claude\u002Fprompt-caching","\u002Fimages\u002Fblog\u002Fprompt-versioning-management\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fprompt-versioning-management\u002Fog.jpg",[60,61,62],"Prompt versioning in production: git, registries, labels and rollbacks","How to version prompts in production: what belongs in a version, git versus a prompt registry, labels and rollbacks, eval gates in CI and cache side effects.","Five upright cards in a row, the third turned back on a curved arrow.",{"slug":560,"published":379,"minutes":561,"category":7,"tags":562,"keywords":567,"about":578,"sources":588,"cover":609,"og":610,"expertise":58,"locales":611,"lang":60,"title":612,"description":613,"coverAlt":614},"llm-batch-api-offline-workloads",9,[563,564,507,565,566],"Batch API","Cost","Prompt caching","Data pipelines",[568,569,570,571,572,573,574,575,576,577],"llm batch api","anthropic message batches api","openai batch api","gemini batch mode","batch inference llm cost","batch api 50 percent discount","openai flex processing","bedrock batch inference","llm offline processing","batch api prompt caching",[579,582,585],{"name":580,"url":581},"Batch processing","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FBatch_processing",{"name":583,"url":584},"JSON Lines","https:\u002F\u002Fjsonlines.org\u002F",{"name":586,"url":587},"Large language model","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FLarge_language_model",[589,592,594,595,598,601,603,606],{"title":590,"url":591},"Claude Platform Docs: Batch processing","https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fbuild-with-claude\u002Fbatch-processing",{"title":593,"url":467},"Claude Platform Docs: Rate limits (Message Batches API)",{"title":551,"url":552},{"title":596,"url":597},"OpenAI API Docs: Batch API","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fguides\u002Fbatch",{"title":599,"url":600},"OpenAI API Docs: Flex processing","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fguides\u002Fflex-processing",{"title":602,"url":473},"OpenAI API Docs: Rate limits (batch queue limits)",{"title":604,"url":605},"Google AI for Developers: Gemini Batch API","https:\u002F\u002Fai.google.dev\u002Fgemini-api\u002Fdocs\u002Fbatch-api",{"title":607,"url":608},"Amazon Bedrock User Guide: Batch inference","https:\u002F\u002Fdocs.aws.amazon.com\u002Fbedrock\u002Flatest\u002Fuserguide\u002Fbatch-inference.html","\u002Fimages\u002Fblog\u002Fllm-batch-api-offline-workloads\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fllm-batch-api-offline-workloads\u002Fog.jpg",[60,61,62],"LLM batch APIs: half-price inference for work that can wait","When to use the Anthropic, OpenAI and Gemini batch APIs, how limits, result matching and failures work, and how batch discounts stack with prompt caching.","A wide box holding eight small squares, tiny squares arriving from the left, one long arrow returning.",1792261156295]