Prompt versioning in production: git, registries, labels and rollbacks

Most LLM features change more often in their prompts than in their code. A support assistant gets a new escalation rule, an extraction step learns about a new invoice layout, a classifier gets one more category. Each of these is a behaviour change shipped to production, and in many codebases it is the least controlled one: a string edited in a dashboard, a template pasted into a config file, no review and no record of which version answered a given customer last Tuesday.

This guide covers what a prompt version should contain, when git is enough and when a prompt registry pays for itself, how labels make rollbacks cheap, how to gate prompt changes with evals in CI, and one side effect teams often miss: a prompt release resets your provider-side prompt cache. It also covers a 2026 change that settles part of the debate, OpenAI retiring its saved prompt objects.

What belongs in a prompt version

A template string alone does not determine behaviour. The same text produces different output with a different model, temperature, tool list or output schema. If your version history records only the text, you cannot answer the question that matters during an incident: what exactly ran? Version the bundle:

Both registries discussed below support this. Langfuse prompts carry a config field next to the template, and MLflow lets each version carry a commit message much like a git commit. In git you get it by keeping the template, parameters and schema in the same module.

Git or a prompt registry?

In 2026 the strongest argument for git came from a vendor that used to sell the alternative. OpenAI is deprecating reusable prompt objects in its API: prompt creation is de-emphasized from 3 June 2026 and v1/prompts is scheduled to shut down on 30 November 2026. Its migration guide tells teams to move prompt content into application code so prompt changes go through the same review and release process as other code, to replace prompt variables with typed function arguments, and to manage versions with commits, pull requests and evals in CI.

Registries still solve real problems. Langfuse and MLflow store immutable versions with movable labels, so a product manager can test a wording change and an on-call engineer can roll back in seconds without a deploy. They also join production metrics to versions. The question is not which camp is right, but who edits prompts and how fast a rollback must be.

Immutable versions, movable labels

The useful mental model is the same one container registries use: versions are immutable, tags move. MLflow states that prompt versions are immutable for reproducibility and that an alias such as production is a mutable reference you can repoint for A/B tests or rollbacks. Langfuse assigns version numbers automatically, maintains a latest label pointing at the newest version, and returns the version labelled production when you fetch without a label or version.

Two details from the Langfuse documentation are worth designing around. Requesting a label that no version carries returns a 404; there is no silent fallback to production or latest. And protected labels, available in its paid and enterprise editions, stop viewer and member roles from moving production, which is the registry equivalent of a protected branch.

Fetching prompts at runtime without new failure modes

A registry puts a network call between your feature and its instructions. Langfuse handles that with a client-side cache: the default TTL is 60 seconds, and when it expires the SDK keeps serving the stale prompt while it refetches in the background. If the cache is empty and the API is unreachable, a fallback prompt you pass in is returned instead. That last case is the one to plan for, because it happens on cold start in exactly the incident where the registry is down.

from langfuse import get_client
from prompts.support import SUPPORT_SYSTEM_FALLBACK  # the same text, committed in git

langfuse = get_client()

def support_prompt():
    # label, not version: promotion and rollback happen by moving "production"
    return langfuse.get_prompt(
        "support-assistant",
        label="production",
        cache_ttl_seconds=300,          # default is 60; longer means slower rollbacks
        fallback=SUPPORT_SYSTEM_FALLBACK,
    )

The TTL is a trade-off you should write down: the longer it is, the fewer registry calls you make and the longer a rollback takes to reach every instance. Pin an explicit version in batch jobs and evals, where reproducibility matters more than following the label.

If prompts live in code, structure them

OpenAI's guide is specific: keep prompts in a small prompts/ module with one named builder per prompt and lightweight eval fixtures, and do not scatter them inline across the codebase. Add two habits. First, put static content first and dynamic content last, which the same guide recommends because cache hits depend on exact prefix matches. Second, export a version identifier with each prompt, so traces can record which one ran even without a registry.

// prompts/invoice-extract.ts
import { z } from 'zod'

export const INVOICE_EXTRACT = {
  id: 'invoice-extract',
  version: '2026-10-14.2',          // bump on any change to text, model, params or schema
  model: 'claude-sonnet-5-5',
  params: { max_tokens: 2048 },
  schema: z.object({ supplier: z.string(), total: z.number(), currency: z.string().length(3) }),
  build: (input: { pdfText: string; locale: 'de' | 'en' | 'hu' }) => ({
    system: SYSTEM_RULES,           // static, long, cacheable: first
    messages: [{ role: 'user' as const, content: `Locale: ${input.locale}\n\n${input.pdfText}` }],
  }),
}

Gate every prompt change with an eval

Versioning tells you what changed; only an evaluation tells you whether it got better. The cheapest workable setup runs a fixed test set against the old and new prompt on every pull request that touches prompt files. The Promptfoo GitHub Action does exactly this: it triggers on changes under your prompt paths, compares prompts before and after the change, posts the result as a PR comment with a link to its viewer, and caches model calls between runs to keep cost down.

# .github/workflows/prompt-eval.yml (trimmed)
on:
  pull_request:
    paths: ['prompts/**']
jobs:
  evaluate:
    runs-on: ubuntu-latest
    permissions: { pull-requests: write }
    steps:
      - uses: actions/checkout@v4
      - uses: promptfoo/promptfoo-action@v1
        with:
          github-token: ${{ secrets.GITHUB_TOKEN }}
          prompts: 'prompts/**/*.json'
          config: 'prompts/promptfooconfig.yaml'
          cache-path: ~/.cache/promptfoo

What goes into the test set matters more than the tool. Start from real failures, as described in LLM evals for product features: every bug report that a prompt change fixed becomes a case, so the next change cannot quietly bring it back. A model upgrade is just a prompt version with a new model id, and gets the same gate, see model migrations without regressions.

Link production traces to prompt versions

Offline evals cover the cases you thought of. Production covers the rest, but only if each generation records which prompt version produced it. Langfuse links a prompt to a generation through the SDK or its OpenAI, LangChain and AI SDK integrations, and its metrics tab then compares versions on median latency, input and output tokens, cost, generation count and median score. One caveat from the documentation: when the fallback prompt is used, no link is created, so a spike of unlinked generations is itself a signal that the registry was unreachable.

Without a registry, put the prompt id and version from your code into the span attributes. The OpenTelemetry setup in observability for LLM agents gives you the same comparison with a group-by.

A prompt release is also a cache event

Provider prompt caches key on exact prefixes. Anthropic caches tools, then system, then messages, and a change at one level invalidates that level and everything after it: edit a tool description and the whole cache is gone, edit the system prompt and system plus messages are rebuilt. Cache writes cost more than normal input (1.25 times base for the five-minute cache) and reads cost a fraction, so right after a release you pay write prices and see longer time to first token until the new prefix is warm. The mechanics are covered in prompt caching and model routing.

A sensible default for small teams

For the projects I build, the default is git first. Prompts live in a prompts/ module with their model, parameters and schema; a version string is bumped on every change; a promptfoo or equivalent eval runs on the pull request; and the version goes into every trace. A registry comes in when someone outside the dev team needs to edit prompts, or when a rollback has to be faster than a deploy. Then git stays the source of truth and CI publishes to the registry, so the label moves only after the eval gate passes.

• Never edit production text in place. Create a version, evaluate it, move the label.
• Pin versions in evals and batch jobs, follow labels only in interactive paths.
• Keep a fallback prompt in the repository for every registry-managed prompt.
• Treat prompts as code in reviews: the diff of a prompt deserves the same attention as the diff of a SQL query.

Sources

1. OpenAI API Docs: Migrate from prompt objects
2. OpenAI API Docs: Prompt engineering (reusable prompts deprecation note)
3. Langfuse Docs: Prompt version control
4. Langfuse Docs: Prompt caching in the SDKs
5. Langfuse Docs: Link prompts to traces
6. MLflow Docs: Prompt Registry
7. Promptfoo Docs: GitHub Action
8. Claude Platform Docs: Prompt caching