> How to version prompts in production: what belongs in a version, git versus a prompt registry, labels and rollbacks, eval gates in CI and cache side effects.
>
> Web page: https://balazscsorba.com/blog/prompt-versioning-management · Language: English · Also available in: [Deutsch](https://balazscsorba.com/de/blog/prompt-versioning-management.md) · [Magyar](https://balazscsorba.com/hu/blog/prompt-versioning-management.md)
> Author: Balázs Csorba · Published: 2026-10-17 · Keywords: prompt versioning, prompt management, prompt registry, version control for prompts, langfuse prompt management, mlflow prompt registry, prompts in git, openai prompt object deprecation, prompt rollback production, prompt ci evaluation

[Blog](https://balazscsorba.com/blog)/LLMOps & evals

# Prompt versioning in production: git, registries, labels and rollbacks

How to version prompts in production: what belongs in a version, git versus a prompt registry, labels and rollbacks, eval gates in CI and cache side effects.

[Balázs Csorba](https://balazscsorba.com/about)·October 17, 2026·8 min read

-   Prompt management
-   Versioning
-   LLMOps
-   Evals
-   Langfuse

![Five upright cards in a row, the third turned back on a curved arrow.](https://balazscsorba.com/images/blog/prompt-versioning-management/cover.webp?v=8cacc8485a)

## Key takeaways

-   A prompt version is the whole bundle: template, model, parameters, tools and output schema. Version them together or you cannot reproduce a result.
-   OpenAI is retiring reusable prompt objects, with v1/prompts scheduled to shut down on 30 November 2026, and now recommends keeping prompts in application code.
-   Registries such as Langfuse and MLflow add immutable versions and movable labels, which make rollbacks a label change instead of a deploy.
-   Gate every prompt change with an eval run on the pull request, and link production traces to the prompt version that produced them.
-   Changing a system prompt or tool definition invalidates provider prompt caches, so a prompt release is also a short cost and latency event.

On this page

1.  [What belongs in a prompt version](https://balazscsorba.com/#what-is-a-version)
2.  [Git or a prompt registry?](https://balazscsorba.com/#git-or-registry)
3.  [Immutable versions, movable labels](https://balazscsorba.com/#versions-and-labels)
4.  [Fetching prompts at runtime without new failure modes](https://balazscsorba.com/#runtime-fetch)
5.  [If prompts live in code, structure them](https://balazscsorba.com/#prompts-in-code)
6.  [Gate every prompt change with an eval](https://balazscsorba.com/#eval-gate)
7.  [Link production traces to prompt versions](https://balazscsorba.com/#link-to-traces)
8.  [A prompt release is also a cache event](https://balazscsorba.com/#cache-side-effects)
9.  [A sensible default for small teams](https://balazscsorba.com/#my-default)
10.  [Sources](https://balazscsorba.com/#sources)

Listen to this article

0:000:00

Most LLM features change more often in their prompts than in their code. A support assistant gets a new escalation rule, an extraction step learns about a new invoice layout, a classifier gets one more category. Each of these is a behaviour change shipped to production, and in many codebases it is the least controlled one: a string edited in a dashboard, a template pasted into a config file, no review and no record of which version answered a given customer last Tuesday.

This guide covers what a prompt version should contain, when git is enough and when a prompt registry pays for itself, how labels make rollbacks cheap, how to gate prompt changes with evals in CI, and one side effect teams often miss: a prompt release resets your provider-side prompt cache. It also covers a 2026 change that settles part of the debate, OpenAI retiring its saved prompt objects.

## What belongs in a prompt version

A template string alone does not determine behaviour. The same text produces different output with a different model, temperature, tool list or output schema. If your version history records only the text, you cannot answer the question that matters during an incident: what exactly ran? Version the bundle:

Part

Why it belongs in the version

System and user templates

The instructions themselves

Model id

A prompt tuned for one model is a different prompt on another

Parameters

Temperature, max tokens, reasoning effort change output

Tool definitions

Descriptions steer tool choice as much as the system prompt

Output schema

Downstream code depends on it

Few-shot examples

Often the largest and most edited part

Both registries discussed below support this. Langfuse prompts carry a config field next to the template, and MLflow lets each version carry a commit message much like a git commit. In git you get it by keeping the template, parameters and schema in the same module.

## Git or a prompt registry?

In 2026 the strongest argument for git came from a vendor that used to sell the alternative. OpenAI is deprecating reusable prompt objects in its API: prompt creation is de-emphasized from 3 June 2026 and `v1/prompts` is scheduled to shut down on 30 November 2026. Its migration guide tells teams to move prompt content into application code so prompt changes go through the same review and release process as other code, to replace prompt variables with typed function arguments, and to manage versions with commits, pull requests and evals in CI.

Registries still solve real problems. [Langfuse](https://balazscsorba.com/tools/langfuse) and MLflow store immutable versions with movable labels, so a product manager can test a wording change and an on-call engineer can roll back in seconds without a deploy. They also join production metrics to versions. The question is not which camp is right, but who edits prompts and how fast a rollback must be.

Criterion

Prompts in git

Prompt registry

Who edits

Developers

Developers and domain experts

Review

Pull request, same as code

Tool-specific, often weaker

Rollback

Revert and deploy

Move a label, effective after cache TTL

Runtime dependency

None

Registry API, mitigated by cache and fallback

Metrics per version

Only if you tag traces yourself

Built in when traces are linked

## Immutable versions, movable labels

The useful mental model is the same one container registries use: versions are immutable, tags move. MLflow states that prompt versions are immutable for reproducibility and that an alias such as `production` is a mutable reference you can repoint for A/B tests or rollbacks. Langfuse assigns version numbers automatically, maintains a `latest` label pointing at the newest version, and returns the version labelled `production` when you fetch without a label or version.

Rolling back means pointing the production label at version 13 again; nothing is rebuilt.

Two details from the Langfuse documentation are worth designing around. Requesting a label that no version carries returns a 404; there is no silent fallback to `production` or `latest`. And protected labels, available in its paid and enterprise editions, stop viewer and member roles from moving `production`, which is the registry equivalent of a protected branch.

## Fetching prompts at runtime without new failure modes

A registry puts a network call between your feature and its instructions. Langfuse handles that with a client-side cache: the default TTL is 60 seconds, and when it expires the SDK keeps serving the stale prompt while it refetches in the background. If the cache is empty and the API is unreachable, a fallback prompt you pass in is returned instead. That last case is the one to plan for, because it happens on cold start in exactly the incident where the registry is down.

```
from langfuse import get_client
from prompts.support import SUPPORT_SYSTEM_FALLBACK  # the same text, committed in git

langfuse = get_client()

def support_prompt():
    # label, not version: promotion and rollback happen by moving "production"
    return langfuse.get_prompt(
        "support-assistant",
        label="production",
        cache_ttl_seconds=300,          # default is 60; longer means slower rollbacks
        fallback=SUPPORT_SYSTEM_FALLBACK,
    )
```

The TTL is a trade-off you should write down: the longer it is, the fewer registry calls you make and the longer a rollback takes to reach every instance. Pin an explicit version in batch jobs and evals, where reproducibility matters more than following the label.

## If prompts live in code, structure them

OpenAI's guide is specific: keep prompts in a small `prompts/` module with one named builder per prompt and lightweight eval fixtures, and do not scatter them inline across the codebase. Add two habits. First, put static content first and dynamic content last, which the same guide recommends because cache hits depend on exact prefix matches. Second, export a version identifier with each prompt, so traces can record which one ran even without a registry.

```
// prompts/invoice-extract.ts
import { z } from 'zod'

export const INVOICE_EXTRACT = {
  id: 'invoice-extract',
  version: '2026-10-14.2',          // bump on any change to text, model, params or schema
  model: 'claude-sonnet-5-5',
  params: { max_tokens: 2048 },
  schema: z.object({ supplier: z.string(), total: z.number(), currency: z.string().length(3) }),
  build: (input: { pdfText: string; locale: 'de' | 'en' | 'hu' }) => ({
    system: SYSTEM_RULES,           // static, long, cacheable: first
    messages: [{ role: 'user' as const, content: `Locale: ${input.locale}\n\n${input.pdfText}` }],
  }),
}
```

## Gate every prompt change with an eval

Versioning tells you what changed; only an evaluation tells you whether it got better. The cheapest workable setup runs a fixed test set against the old and new prompt on every pull request that touches prompt files. The [Promptfoo](https://balazscsorba.com/tools/promptfoo) GitHub Action does exactly this: it triggers on changes under your prompt paths, compares prompts before and after the change, posts the result as a PR comment with a link to its viewer, and caches model calls between runs to keep cost down.

```
# .github/workflows/prompt-eval.yml (trimmed)
on:
  pull_request:
    paths: ['prompts/**']
jobs:
  evaluate:
    runs-on: ubuntu-latest
    permissions: { pull-requests: write }
    steps:
      - uses: actions/checkout@v4
      - uses: promptfoo/promptfoo-action@v1
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          prompts: 'prompts/**/*.json'
          config: 'prompts/promptfooconfig.yaml'
          cache-path: ~/.cache/promptfoo
```

What goes into the test set matters more than the tool. Start from real failures, as described in [LLM evals for product features](https://balazscsorba.com/blog/llm-evals-for-product-features): every bug report that a prompt change fixed becomes a case, so the next change cannot quietly bring it back. A model upgrade is just a prompt version with a new model id, and gets the same gate, see [model migrations without regressions](https://balazscsorba.com/blog/llm-model-migration-regression-testing).

## Link production traces to prompt versions

Offline evals cover the cases you thought of. Production covers the rest, but only if each generation records which prompt version produced it. Langfuse links a prompt to a generation through the SDK or its OpenAI, LangChain and AI SDK integrations, and its metrics tab then compares versions on median latency, input and output tokens, cost, generation count and median score. One caveat from the documentation: when the fallback prompt is used, no link is created, so a spike of unlinked generations is itself a signal that the registry was unreachable.

Without a registry, put the prompt id and version from your code into the span attributes. The OpenTelemetry setup in [observability for LLM agents](https://balazscsorba.com/blog/agent-observability-opentelemetry) gives you the same comparison with a group-by.

## A prompt release is also a cache event

Provider prompt caches key on exact prefixes. Anthropic caches `tools`, then `system`, then `messages`, and a change at one level invalidates that level and everything after it: edit a tool description and the whole cache is gone, edit the system prompt and system plus messages are rebuilt. Cache writes cost more than normal input (1.25 times base for the five-minute cache) and reads cost a fraction, so right after a release you pay write prices and see longer time to first token until the new prefix is warm. The mechanics are covered in [prompt caching and model routing](https://balazscsorba.com/blog/llm-cost-latency-prompt-caching-routing).

**Keep volatile text out of the prefix**

Dates, user names, feature flags and A/B variant text belong at the end of the prompt, not in the system prompt. Otherwise every variant and every day is a separate cache prefix, and a harmless experiment doubles your input cost.

## A sensible default for small teams

For the projects I build, the default is git first. Prompts live in a `prompts/` module with their model, parameters and schema; a version string is bumped on every change; a promptfoo or equivalent eval runs on the pull request; and the version goes into every trace. A registry comes in when someone outside the dev team needs to edit prompts, or when a rollback has to be faster than a deploy. Then git stays the source of truth and CI publishes to the registry, so the label moves only after the eval gate passes.

-   **Never edit production text in place.** Create a version, evaluate it, move the label.
-   **Pin versions in evals and batch jobs**, follow labels only in interactive paths.
-   **Keep a fallback prompt in the repository** for every registry-managed prompt.
-   **Treat prompts as code in reviews:** the diff of a prompt deserves the same attention as the diff of a SQL query.

## Sources

1.  [OpenAI API Docs: Migrate from prompt objects](https://developers.openai.com/api/docs/guides/prompting/migrate-from-prompt-object)
2.  [OpenAI API Docs: Prompt engineering (reusable prompts deprecation note)](https://developers.openai.com/api/docs/guides/prompt-engineering)
3.  [Langfuse Docs: Prompt version control](https://langfuse.com/docs/prompt-management/features/prompt-version-control)
4.  [Langfuse Docs: Prompt caching in the SDKs](https://langfuse.com/docs/prompt-management/features/caching)
5.  [Langfuse Docs: Link prompts to traces](https://langfuse.com/docs/prompt-management/features/link-to-traces)
6.  [MLflow Docs: Prompt Registry](https://mlflow.org/docs/latest/genai/prompt-registry/)
7.  [Promptfoo Docs: GitHub Action](https://www.promptfoo.dev/docs/integrations/github-action/)
8.  [Claude Platform Docs: Prompt caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching)

## Frequently asked questions

Should prompts live in git or in a prompt management tool?

Put them in git by default: review, history and tests come for free, and OpenAI now recommends exactly that. A registry is worth it when non-developers edit prompts, when you need to roll back without a deploy, or when you want production metrics per version. Many teams do both: git as the source, a registry as the runtime store.

What is the difference between a prompt version and a label?

A version is an immutable snapshot of the prompt; MLflow states that versions cannot be edited after creation. A label or alias such as production is a movable pointer to one version. Your code asks for the label, and promoting or rolling back means moving the pointer.

What happens to OpenAI saved prompts?

OpenAI is deprecating reusable prompt objects in the API. Prompt creation is de-emphasized from 3 June 2026 and the v1/prompts endpoint is scheduled to shut down on 30 November 2026. The migration guide says to move prompt text into your code, replace variables with typed function arguments and pass the result as input to the Responses API.

Does fetching prompts from a registry add latency?

Not much if the SDK caches. Langfuse caches prompts client-side with a default TTL of 60 seconds and refreshes them in the background, serving the stale version meanwhile. If the cache is empty and the API is down, a fallback prompt you pass in keeps the application running.

How do I test a prompt change before it reaches users?

Run the same evaluation set against the old and new prompt on every pull request that touches prompt files. The promptfoo GitHub Action does this and posts the comparison as a PR comment. After release, compare scores, latency and cost per prompt version on linked traces.

Written by Balázs Csorba

Senior fullstack & AI engineer in Styria, Austria – 10+ years of Vue, Nuxt, Node.js and PHP, now building tooling for AI agents.

[AI engineering & MCP servers →](https://balazscsorba.com/expertise/ai-engineer)[About me →](https://balazscsorba.com/about)

## More articles

-   [LLM rate limits in production: retries, backoff and provider fallback](https://balazscsorba.com/blog/llm-rate-limits-retries-provider-fallback)
-   [LLM batch APIs: half-price inference for work that can wait](https://balazscsorba.com/blog/llm-batch-api-offline-workloads)
-   [Structured outputs with JSON Schema: what the guarantee covers, and what it does not](https://balazscsorba.com/blog/structured-outputs-json-schema-llm)
-   [Evaluating AI agents: grading trajectories, outcomes and reliability](https://balazscsorba.com/blog/evaluating-ai-agents-trajectories)

## Sounds like what you need?

Tell me about your project or role – I’d love to hear from you.

[Get in touch](mailto:contact@balazscsorba.com) [Connect on LinkedIn](https://www.linkedin.com/in/balazs-csorba)