[{"data":1,"prerenderedAt":807},["ShallowReactive",2],{"blog-llm-output-guardrails-moderation-en":3},{"slug":4,"published":5,"minutes":6,"category":7,"tags":8,"keywords":14,"about":27,"sources":35,"cover":66,"og":67,"expertise":68,"locales":69,"lang":70,"title":73,"description":74,"coverAlt":75,"metaTitle":76,"takeaways":77,"faq":83,"toc":99,"blocks":133,"others":525},"llm-output-guardrails-moderation","2026-10-17",13,"security",[9,10,11,12,13],"LLM security","Guardrails","Content moderation","OWASP","Output validation",[15,16,17,18,19,20,21,22,23,24,25,26],"llm output guardrails","improper output handling llm","owasp llm05","openai moderation api","omni-moderation-latest","llama guard 4","nemo guardrails streaming","markdown image exfiltration","sanitize llm output html","groundedness detection","llm content moderation production","output rails",[28,30,33],{"name":11,"url":29},"https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FContent_moderation",{"name":31,"url":32},"Cross-site scripting","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FCross-site_scripting",{"name":12,"url":34},"https:\u002F\u002Fowasp.org\u002F",[36,39,42,45,48,51,54,57,60,63],{"title":37,"url":38},"OWASP GenAI Security Project: LLM05:2025 Improper Output Handling","https:\u002F\u002Fgenai.owasp.org\u002Fllmrisk\u002Fllm052025-improper-output-handling\u002F",{"title":40,"url":41},"OpenAI API Docs: Moderation","https:\u002F\u002Fplatform.openai.com\u002Fdocs\u002Fguides\u002Fmoderation",{"title":43,"url":44},"Meta: Llama Guard 4 model card and prompt formats","https:\u002F\u002Fwww.llama.com\u002Fdocs\u002Fmodel-cards-and-prompt-formats\u002Fllama-guard-4\u002F",{"title":46,"url":47},"NVIDIA NeMo Guardrails: Output rail streaming configuration","https:\u002F\u002Fdocs.nvidia.com\u002Fnemo\u002Fguardrails\u002Fconfigure-guardrails\u002Fyaml-schema\u002Fstreaming\u002Foutput-rail-streaming.md",{"title":49,"url":50},"Microsoft Learn: Prompt Shields in Azure AI Content Safety","https:\u002F\u002Flearn.microsoft.com\u002Fen-us\u002Fazure\u002Fai-services\u002Fcontent-safety\u002Fconcepts\u002Fjailbreak-detection",{"title":52,"url":53},"Microsoft Learn: Groundedness detection in Azure AI Content Safety","https:\u002F\u002Flearn.microsoft.com\u002Fen-us\u002Fazure\u002Fai-services\u002Fcontent-safety\u002Fconcepts\u002Fgroundedness",{"title":55,"url":56},"Microsoft Learn: Protected material detection in Azure AI Content Safety","https:\u002F\u002Flearn.microsoft.com\u002Fen-us\u002Fazure\u002Fai-services\u002Fcontent-safety\u002Fconcepts\u002Fprotected-material",{"title":58,"url":59},"Guardrails AI documentation","https:\u002F\u002Fwww.guardrailsai.com\u002Fdocs",{"title":61,"url":62},"Vue.js docs: Security best practices","https:\u002F\u002Fvuejs.org\u002Fguide\u002Fbest-practices\u002Fsecurity.html",{"title":64,"url":65},"Simon Willison: Remote prompt injection in GitLab Duo leads to source code theft","https:\u002F\u002Fsimonwillison.net\u002F2025\u002FMay\u002F23\u002Fremote-prompt-injection-in-gitlab-duo\u002F","\u002Fimages\u002Fblog\u002Fllm-output-guardrails-moderation\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fllm-output-guardrails-moderation\u002Fog.jpg","ai-engineer",[70,71,72],"en","de","hu","LLM output guardrails: moderation, safe rendering and validation before anything acts","What to check after the model answers: safe Markdown rendering, schema checks before actions, moderation classifiers, streaming rails and groundedness.","Three narrowing rectangular frames stacked as a funnel, dots falling through to a tray beside a lock.","LLM output guardrails and moderation · Balázs Csorba",[78,79,80,81,82],"OWASP LLM05:2025 treats model output as untrusted input to the next system: browser, database, shell or mail template.","Most real output incidents are rendering and action bugs, not offensive text, so sanitise Markdown and lock down images and links first.","Validate structured output against a schema and an allowlist before it triggers any action; never pass it to eval, a shell or string-built SQL.","Moderation classifiers such as omni-moderation-latest or Llama Guard 4 give signals, not verdicts; you choose thresholds and actions per category.","Streaming changes the trade-off: either the user may see a chunk before it is blocked, or you pay the check latency on every chunk.",[84,87,90,93,96],{"q":85,"a":86},"What is improper output handling?","It is the OWASP LLM Top 10 entry LLM05:2025: insufficient validation, sanitisation and handling of model output before it is passed to other components. Typical results are XSS in the browser, SQL injection, path traversal or remote code execution on the backend.",{"q":88,"a":89},"Is the OpenAI moderation endpoint enough as an output guardrail?","It covers content-safety categories such as hate, harassment, self-harm, sexual content, violence and illicit advice, and it is free to use. It does not check whether HTML is safe to render, whether an answer is grounded, or whether a tool call is allowed. Treat it as one layer.",{"q":91,"a":92},"Can I moderate a streamed response?","Yes, but you have to choose. OpenAI returns inline moderation scores only after the full output is available. NeMo Guardrails checks chunks of 200 tokens by default and either streams each chunk before the check or holds it until the check passes.",{"q":94,"a":95},"Should I block or rewrite flagged output?","It depends on the category. Block and replace with a fixed message for clear safety violations, redact for leaked personal data or secrets, and route borderline or business-critical cases to a person. Log every decision with the scores so you can tune thresholds.",{"q":97,"a":98},"How do I stop Markdown image exfiltration?","Do not render images or links to arbitrary domains from model output. Use an allowlist of hosts, strip or rewrite everything else on the server, disable raw HTML in the Markdown renderer and add a strict Content Security Policy.",[100,103,106,109,112,115,118,121,124,127,130],{"id":101,"title":102},"output-is-untrusted-input","Model output is untrusted input to the next system",{"id":104,"title":105},"four-jobs","Four different jobs that all get called guardrails",{"id":107,"title":108},"render-safely","Render Markdown as if an attacker wrote it",{"id":110,"title":111},"validate-before-acting","Validate structured output before it does anything",{"id":113,"title":114},"moderation-classifiers","Moderation classifiers: what the main options check",{"id":116,"title":117},"streaming","Streaming: see it first or check it first",{"id":119,"title":120},"groundedness-and-leakage","Groundedness and leakage checks",{"id":122,"title":123},"actions-and-thresholds","Block, redact, rewrite or escalate",{"id":125,"title":126},"layered-pipeline","Putting it together in one server route",{"id":128,"title":129},"testing-guardrails","Testing guardrails like any other feature",{"id":131,"title":132},"sources","Sources",[134,144,147,150,162,165,166,169,207,210,211,214,229,231,237,238,246,267,274,275,278,308,318,328,335,336,351,368,373,374,381,389,390,393,437,444,445,448,455,457,468,469,472,487,490,491],{"type":135,"content":136},"paragraph",[137,138,143],"Most guardrail discussions start at the input: prompt injection, jailbreaks, people asking for things they should not get. That half matters, and I wrote about it in ",{"tag":139,"to":140,"children":141},"link","\u002Fblog\u002Fprompt-injection-lethal-trifecta-patterns",[142],"prompt injection defense",". This post is about the other half. Once the model has answered, its text goes somewhere: into a chat bubble rendered as Markdown, into a JSON parser that triggers a refund, into an email template, into a SQL query. Every one of those destinations has its own rules about what is safe, and the model knows none of them.",{"type":135,"content":145},[146],"The OWASP Top 10 for LLM applications names this risk LLM05:2025 Improper Output Handling. Below is how I structure output checks in production features: what to render and how, what to validate before an action, where moderation classifiers fit, what streaming does to all of this, and how to decide between blocking, rewriting and escalating.",{"type":148,"level":149,"id":101,"text":102},"heading",2,{"type":135,"content":151},[152,153,157,158,161],"OWASP defines improper output handling as insufficient validation, sanitisation and handling of model output before it is passed downstream. The examples in the entry are ordinary web vulnerabilities with a new source: output passed to ",{"tag":154,"children":155},"code",[156],"exec"," or ",{"tag":154,"children":159},[160],"eval",", JavaScript or Markdown interpreted by the browser as XSS, generated SQL executed without parameterisation, file paths built from output, and content placed in email templates without escaping.",{"type":135,"content":163},[164],"The reason this deserves its own category is that the attacker does not need access to your output. With indirect prompt injection, a document, product review or web page the model reads can steer what it writes. OWASP lists exactly that as a factor that raises the impact, next to model privileges beyond those of the end user and missing context-aware encoding. Its first mitigation is blunt: treat the model as any other user and apply zero trust to what it returns.",{"type":148,"level":149,"id":104,"text":105},{"type":135,"content":167},[168],"Part of the confusion around output guardrails is that the word covers four jobs with different tools and different owners. I keep them apart in design reviews, because a team that bought a moderation API often believes it has covered all four.",{"type":170,"head":171,"rows":178},"table",[172,174,176],[173],"Job",[175],"Question it answers",[177],"Typical tool",[179,186,193,200],[180,182,184],[181],"Safe rendering",[183],"Can this text run code or leak data when displayed?",[185],"Markdown renderer config, sanitiser, CSP",[187,189,191],[188],"Action validation",[190],"Is this tool call or JSON allowed and well-formed?",[192],"JSON Schema, allowlists, permission checks",[194,196,198],[195],"Content safety",[197],"Is this harmful, hateful or against policy?",[199],"omni-moderation-latest, Llama Guard 4, Azure AI Content Safety",[201,203,205],[202],"Correctness and leakage",[204],"Is it grounded, and does it expose data it should not?",[206],"Groundedness checks, PII and secret scanners",{"type":135,"content":208},[209],"The order of that table is also the order in which I would fund the work. Rendering and action bugs turn a single bad answer into an incident; offensive text in a B2B support widget is embarrassing, but it rarely deletes data.",{"type":148,"level":149,"id":107,"text":108},{"type":135,"content":212},[213],"Chat interfaces render Markdown because it looks good. Markdown is also a way to make the browser fetch URLs. The classic exfiltration pattern goes like this: an injected instruction asks the model to put private data into the query string of an image URL, and the browser sends it to the attacker the moment the image renders, without any click. Simon Willison has catalogued it across many products; in the GitLab Duo case reported in May 2025, a hidden instruction in a public project made the assistant fetch a private merge request and exfiltrate a base64 summary through an image tag. GitLab fixed it by allowing links and images only to trusted domains.",{"type":135,"content":215},[216,217,220,221,224,225,228],"In a Vue or Nuxt front end the rules are the ones the Vue security guide already gives for user content: never use untrusted content as a template, avoid ",{"tag":154,"children":218},[219],"v-html"," with unsanitised HTML, and sanitise URLs on the server. For model output I do four things: disable raw HTML in the Markdown renderer, rewrite or drop images and links whose host is not on an allowlist, add ",{"tag":154,"children":222},[223],"rel=\"noopener nofollow\""," to remaining external links, and send a Content Security Policy whose ",{"tag":154,"children":226},[227],"img-src"," lists only my own hosts. The CSP is the backstop that still works when the renderer has a bug.",{"type":154,"code":230},"\u002F\u002F server\u002Futils\u002Frender-answer.ts — Markdown from the model, rendered on the server\nimport MarkdownIt from 'markdown-it'\n\nconst ALLOWED_HOSTS = new Set(['balazscsorba.com', 'cdn.example-shop.at'])\nconst md = new MarkdownIt({ html: false, linkify: false })\n\nfunction allowed(url: string) {\n  try {\n    const u = new URL(url, 'https:\u002F\u002Fbalazscsorba.com')\n    return (u.protocol === 'https:' && ALLOWED_HOSTS.has(u.hostname))\n  } catch { return false }\n}\n\n\u002F\u002F images to unknown hosts become plain alt text, so nothing is fetched\nmd.renderer.rules.image = (tokens, i) => {\n  const src = tokens[i].attrGet('src') ?? ''\n  return allowed(src) ? `\u003Cimg src=\"${md.utils.escapeHtml(src)}\" alt=\"${md.utils.escapeHtml(tokens[i].content)}\">`\n                      : md.utils.escapeHtml(tokens[i].content)\n}\nconst defaultLink = md.renderer.rules.link_open ?? ((t, i, o, e, s) => s.renderToken(t, i, o))\nmd.renderer.rules.link_open = (tokens, i, opts, env, self) => {\n  if (!allowed(tokens[i].attrGet('href') ?? '')) tokens[i].attrSet('href', '#blocked-link')\n  tokens[i].attrSet('rel', 'noopener nofollow')\n  return defaultLink(tokens, i, opts, env, self)\n}\n\nexport const renderAnswer = (text: string) => md.render(text)\n",{"type":135,"content":232},[233,234,236],"Render on the server and ship HTML to the client only after these rules ran. If you render in the browser, apply the same rules there and still keep the CSP, because a single component that falls back to ",{"tag":154,"children":235},[219]," on raw output undoes the rest.",{"type":148,"level":149,"id":110,"text":111},{"type":135,"content":239},[240,241,245],"When output drives an action, a tool call, a refund, a database query, the checks are deterministic and cheap, so there is no excuse to skip them. Strict JSON Schema mode guarantees shape, not meaning, as I explained in ",{"tag":139,"to":242,"children":243},"\u002Fblog\u002Fstructured-outputs-json-schema-llm",[244],"structured outputs with JSON Schema",". After parsing, check the values against business rules: is this order number one the current user may see, is this refund below the limit the assistant may grant on its own, is this SKU in the catalogue.",{"type":247,"items":248},"list",[249,254,256,258,260],[250,251,253],"Never pass output to ",{"tag":154,"children":252},[160],", a shell, or a template engine that executes expressions.",[255],"Use parameterised queries for every value that came from the model; OWASP lists this explicitly.",[257],"Resolve file names and IDs against a lookup table instead of building paths from text.",[259],"Escape output for the context it lands in: HTML, email, CSV, a shell argument.",[261,262,266],"Put a person in front of irreversible actions, as in ",{"tag":139,"to":263,"children":264},"\u002Fblog\u002Fhuman-in-the-loop-ai-agents",[265],"human-in-the-loop approval gates",".",{"type":135,"content":268},[269,270,266],"For generated SQL in particular, the read-only role and the query validator matter more than any classifier; the details are in ",{"tag":139,"to":271,"children":272},"\u002Fblog\u002Ftext-to-sql-llm-guardrails",[273],"text-to-SQL guardrails",{"type":148,"level":149,"id":113,"text":114},{"type":135,"content":276},[277],"For content safety you rarely write your own model. Three families cover most projects I see: the OpenAI moderation endpoint, Meta Llama Guard 4 as an open-weights model you host or rent, and Azure AI Content Safety with several specialised detectors.",{"type":170,"head":279,"rows":286},[280,282,284],[281],"Option",[283],"What it classifies",[285],"Notes from the docs",[287,294,301],[288,290,292],[289],"OpenAI omni-moderation-latest",[291],"Text and images; categories such as harassment, hate, illicit, self-harm, sexual, violence",[293],"Free to use. Many categories are text only. Can run inline in a generation request.",[295,297,299],[296],"Meta Llama Guard 4 (12B)",[298],"Prompts and responses against hazard categories S1 to S14",[300],"Open weights, text and images. Outputs safe or unsafe plus the violated categories. Categories can be customised in the prompt.",[302,304,306],[303],"Azure AI Content Safety",[305],"Harm categories, Prompt Shields, groundedness, protected material",[307],"Protected material detection is English only; the code index is current only through April 2023.",{"type":135,"content":309},[310,311,157,314,317],"Llama Guard 4 uses the MLCommons hazard taxonomy plus one addition: S1 violent crimes, S2 non-violent crimes, S3 sex crimes, S4 child exploitation, S5 defamation, S6 specialized advice, S7 privacy, S8 intellectual property, S9 indiscriminate weapons, S10 hate, S11 self-harm, S12 sexual content, S13 elections and S14 code interpreter abuse. To check a response, Meta says the prompt must contain both the user message and the agent reply, because the question is context for judging the answer. Its first output line is ",{"tag":154,"children":312},[313],"safe",{"tag":154,"children":315},[316],"unsafe",", the second lists the violated categories.",{"type":135,"content":319},[320,321,324,325,327],"OpenAI added inline moderation: pass a top-level ",{"tag":154,"children":322},[323],"moderation"," object with ",{"tag":154,"children":326},[19]," in a Responses or Chat Completions request and you get scores for both input and output, without a second call. The docs are explicit about two things I would repeat to every team. A safety-aware refusal can still be flagged because it discusses harmful content, so treat scores as signals for your policy, not as an automatic block. And for tool calls, moderation covers arguments and outputs in the conversation, not tool names, descriptions or schemas.",{"type":329,"variant":330,"title":331,"body":332},"callout","warn","A moderation API is not a security boundary",[333],[334],"None of these classifiers checks whether an image URL leaks data, whether a JSON refund is within limits, or whether the answer is true. A clean moderation result on an exfiltration payload is the expected outcome, not a failure of the classifier.",{"type":148,"level":149,"id":116,"text":117},{"type":135,"content":337},[338,339,342,343,346,347,350],"Users expect tokens to appear as they are generated, and every output check wants the whole text. You cannot have both, and the tools say so. OpenAI states that inline moderation scores arrive only after the full output is available, not with the deltas. NVIDIA NeMo Guardrails runs output rails on chunks when streaming is enabled: ",{"tag":154,"children":340},[341],"chunk_size"," defaults to 200 tokens, ",{"tag":154,"children":344},[345],"context_size"," carries 50 tokens over between chunks so a sentence that spans a boundary is still judged as a whole, and ",{"tag":154,"children":348},[349],"stream_first"," decides the order.",{"type":352,"attrs":353,"inner":357,"caption":358},"diagram",{"viewBox":354,"role":355,"aria-labelledby":356},"0 0 720 170","img","ogr-stream-t ogr-stream-d","\u003Ctitle id=\"ogr-stream-t\">Two ways to stream with output checks\u003C\u002Ftitle>\u003Cdesc id=\"ogr-stream-d\">Tokens are buffered into chunks; either the chunk is sent and then checked, or checked and then sent.\u003C\u002Fdesc>\u003Cdefs>\u003Cmarker id=\"ogr-stream-ah\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto-start-reverse\">\u003Cpath d=\"M0 0L10 5L0 10z\" class=\"d-head\" \u002F>\u003C\u002Fmarker>\u003C\u002Fdefs>\u003Crect x=\"20\" y=\"47\" width=\"154\" height=\"76\" rx=\"10\" class=\"d-box\" \u002F>\u003Ctext x=\"97\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Model\u003C\u002Ftext>\u003Ctext x=\"97\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">token stream\u003C\u002Ftext>\u003Cpath d=\"M174 85H194\" class=\"d-line\" marker-end=\"url(#ogr-stream-ah)\" \u002F>\u003Crect x=\"196\" y=\"47\" width=\"154\" height=\"76\" rx=\"10\" class=\"d-box\" \u002F>\u003Ctext x=\"272\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Buffer\u003C\u002Ftext>\u003Ctext x=\"272\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">200-token chunk\u003C\u002Ftext>\u003Cpath d=\"M349 85H369\" class=\"d-line\" marker-end=\"url(#ogr-stream-ah)\" \u002F>\u003Crect x=\"371\" y=\"47\" width=\"154\" height=\"76\" rx=\"10\" class=\"d-fill-accent\" \u002F>\u003Ctext x=\"448\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Rail check\u003C\u002Ftext>\u003Ctext x=\"448\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">+50 tokens context\u003C\u002Ftext>\u003Cpath d=\"M524 85H544\" class=\"d-line\" marker-end=\"url(#ogr-stream-ah)\" \u002F>\u003Crect x=\"546\" y=\"47\" width=\"154\" height=\"76\" rx=\"10\" class=\"d-box\" \u002F>\u003Ctext x=\"623\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Client\u003C\u002Ftext>\u003Ctext x=\"623\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">show or stop\u003C\u002Ftext>",[359,360,363,364,367],"With ",{"tag":154,"children":361},[362],"stream_first: True"," the client gets each chunk before the rail runs; with ",{"tag":154,"children":365},[366],"False"," the rail runs first and adds its latency to every chunk.",{"type":135,"content":369},[359,370,372],{"tag":154,"children":371},[349]," set to true, which is the default, the user may already have read a chunk that is then blocked, and the stream ends with a JSON error your front end has to handle, ideally by replacing the bubble rather than leaving half a sentence. With false, nothing blocked is ever shown, but each chunk waits for the rail. My rule: stream first for low-risk assistants where the worst case is an awkward sentence; check first when output can contain personal data, prices or commitments the business would have to honour.",{"type":148,"level":149,"id":119,"text":120},{"type":135,"content":375},[376,377,266],"In a RAG assistant the most expensive output failure is a confident answer that the sources do not support. Azure AI Content Safety offers groundedness detection for this: you send the generated text and the grounding sources, choose a domain such as Medical or Generic, and it flags ungrounded content. An optional correction mode returns rewritten text aligned with the sources. I use checks like this as a sampling and alerting tool rather than a hard gate, because a grounding judge is itself a model with errors, and the real fix belongs upstream in retrieval and citations, covered in ",{"tag":139,"to":378,"children":379},"\u002Fblog\u002Fllm-hallucination-grounding-citations",[380],"grounding and citations",{"type":135,"content":382},[383,384,388],"Leakage is easier to check deterministically. Scan output for the patterns you know: IBANs, email addresses and phone numbers of people other than the current user, API keys, internal hostnames, and fragments of your own system prompt. The same detectors you use for ",{"tag":139,"to":385,"children":386},"\u002Fblog\u002Fpii-redaction-llm-pipelines",[387],"PII redaction"," on the input side work on the output side; the difference is that on output you usually redact and log instead of rejecting the whole answer.",{"type":148,"level":149,"id":122,"text":123},{"type":135,"content":391},[392],"A guardrail is a decision, not a score. For each category write down what happens when it fires, and who sees it. This is the table I start with and then tune with real traffic.",{"type":170,"head":394,"rows":401},[395,397,399],[396],"Finding",[398],"Action",[400],"User sees",[402,409,416,423,430],[403,405,407],[404],"Unsafe link or image host",[406],"Rewrite to text, log",[408],"Answer without the link",[410,412,414],[411],"Clear content-safety violation",[413],"Block, replace with fixed text, log scores",[415],"Neutral refusal and a contact option",[417,419,421],[418],"PII or secret of someone else",[420],"Redact, alert security",[422],"Answer with the value masked",[424,426,428],[425],"Invalid tool arguments",[427],"Reject, return error to the model once",[429],"Nothing, unless the retry fails",[431,433,435],[432],"Ungrounded claim in a RAG answer",[434],"Flag for review, sample",[436],"Answer with sources, as usual",{"type":135,"content":438},[439,440,443],"Two habits keep this honest. Log the classifier version, scores and the action for every decision, because thresholds drift when the provider updates a model behind a ",{"tag":154,"children":441},[442],"-latest"," alias. And show users something useful when you block: a fixed message and a way to reach a person beats a silent empty bubble, which people read as a bug and retry.",{"type":148,"level":149,"id":125,"text":126},{"type":135,"content":446},[447],"Cheap deterministic checks run first, classifiers second, and rendering last, so that the expensive and fallible steps see as little as possible. In a Nuxt or any Node backend, the non-streaming version is a short function.",{"type":352,"attrs":449,"inner":452,"caption":453},{"viewBox":450,"role":355,"aria-labelledby":451},"0 0 720 180","ogr-pipe-t ogr-pipe-d","\u003Ctitle id=\"ogr-pipe-t\">Output guardrail pipeline\u003C\u002Ftitle>\u003Cdesc id=\"ogr-pipe-d\">Model output passes leakage scan, schema check, moderation and safe rendering before it reaches the user.\u003C\u002Fdesc>\u003Cdefs>\u003Cmarker id=\"ogr-pipe-ah\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto-start-reverse\">\u003Cpath d=\"M0 0L10 5L0 10z\" class=\"d-head\" \u002F>\u003C\u002Fmarker>\u003C\u002Fdefs>\u003Crect x=\"20\" y=\"52\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-box\" \u002F>\u003Ctext x=\"79\" y=\"86\" text-anchor=\"middle\" class=\"d-text\">Output\u003C\u002Ftext>\u003Ctext x=\"79\" y=\"108\" text-anchor=\"middle\" class=\"d-small\">text or JSON\u003C\u002Ftext>\u003Cpath d=\"M138 90H158\" class=\"d-line\" marker-end=\"url(#ogr-pipe-ah)\" \u002F>\u003Crect x=\"160\" y=\"52\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-box\" \u002F>\u003Ctext x=\"220\" y=\"86\" text-anchor=\"middle\" class=\"d-text\">Leak scan\u003C\u002Ftext>\u003Ctext x=\"220\" y=\"108\" text-anchor=\"middle\" class=\"d-small\">PII, secrets\u003C\u002Ftext>\u003Cpath d=\"M279 90H299\" class=\"d-line\" marker-end=\"url(#ogr-pipe-ah)\" \u002F>\u003Crect x=\"301\" y=\"52\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-box\" \u002F>\u003Ctext x=\"360\" y=\"86\" text-anchor=\"middle\" class=\"d-text\">Schema\u003C\u002Ftext>\u003Ctext x=\"360\" y=\"108\" text-anchor=\"middle\" class=\"d-small\">rules, allowlist\u003C\u002Ftext>\u003Cpath d=\"M419 90H439\" class=\"d-line\" marker-end=\"url(#ogr-pipe-ah)\" \u002F>\u003Crect x=\"441\" y=\"52\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-fill-accent\" \u002F>\u003Ctext x=\"500\" y=\"86\" text-anchor=\"middle\" class=\"d-text\">Moderation\u003C\u002Ftext>\u003Ctext x=\"500\" y=\"108\" text-anchor=\"middle\" class=\"d-small\">scores, policy\u003C\u002Ftext>\u003Cpath d=\"M560 90H580\" class=\"d-line\" marker-end=\"url(#ogr-pipe-ah)\" \u002F>\u003Crect x=\"582\" y=\"52\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-box\" \u002F>\u003Ctext x=\"641\" y=\"86\" text-anchor=\"middle\" class=\"d-text\">Render\u003C\u002Ftext>\u003Ctext x=\"641\" y=\"108\" text-anchor=\"middle\" class=\"d-small\">hosts, CSP\u003C\u002Ftext>",[454],"Order by cost: deterministic checks first, the classifier call second, rendering rules last.",{"type":154,"code":456},"\u002F\u002F server\u002Futils\u002Fguard-output.ts\nimport { renderAnswer } from '.\u002Frender-answer'\nimport { redactLeaks } from '.\u002Fredact'          \u002F\u002F your PII \u002F secret patterns\n\nconst BLOCKED = 'I cannot help with that here. Please contact our support team.'\n\nexport async function guardOutput(text: string) {\n  const { text: clean, findings } = redactLeaks(text)\n\n  const res = await fetch('https:\u002F\u002Fapi.openai.com\u002Fv1\u002Fmoderations', {\n    method: 'POST',\n    headers: { Authorization: `Bearer ${process.env.OPENAI_API_KEY}`, 'Content-Type': 'application\u002Fjson' },\n    body: JSON.stringify({ model: 'omni-moderation-latest', input: clean }),\n  })\n  if (!res.ok) throw new Error(`moderation failed: ${res.status}`)   \u002F\u002F fail closed\n  const result = (await res.json()).results[0]\n\n  const decision = result.flagged ? 'block' : findings.length ? 'redact' : 'allow'\n  console.info(JSON.stringify({ event: 'output_guard', decision, findings, scores: result.category_scores }))\n\n  return { html: renderAnswer(decision === 'block' ? BLOCKED : clean), decision }\n}\n",{"type":135,"content":458},[459,460,463,464,266],"Two choices in that sketch are deliberate. It fails closed when the moderation call errors, which is right for a public assistant and wrong for an internal tool where availability matters more; decide it explicitly. And it uses ",{"tag":154,"children":461},[462],"flagged"," as the first decision, as OpenAI suggests, while logging every score so you can later move to per-category thresholds. If you prefer a framework, Guardrails AI wraps the same idea in validators you attach to a model call, which I compared in the ",{"tag":139,"to":465,"children":466},"\u002Ftools\u002Fguardrails-ai",[467],"Guardrails AI review",{"type":148,"level":149,"id":128,"text":129},{"type":135,"content":470},[471],"Guardrails fail in two directions, and both cost money. Missed violations are the obvious one. False positives are the quiet one: a support assistant that refuses questions about \"killing a process\" or \"dangerous goods shipping\" loses users without anyone noticing. Build a small labelled set with both kinds, including domain vocabulary that trips classifiers, and run it in CI whenever you change a threshold, a prompt or a classifier version.",{"type":247,"items":473},[474,476,478,480],[475],"Include exfiltration payloads: image URLs with query strings, links to look-alike domains, HTML tags in Markdown.",[477],"Include tool calls with out-of-range values and IDs that belong to another customer.",[479],"Track block rate and false-positive rate per category on production samples, not only in the test set.",[481,482,486],"Red-team the whole chain with a tool such as ",{"tag":139,"to":483,"children":484},"\u002Ftools\u002Fpromptfoo",[485],"Promptfoo",", since a payload that passes one layer is only interesting if it passes all of them.",{"type":135,"content":488},[489],"Output guardrails are unglamorous engineering: renderer configuration, schemas, a classifier call and a decision table. That is exactly why they work. The model will keep surprising you; the systems after it should not.",{"type":148,"level":149,"id":131,"text":132},{"type":247,"ordered":492,"items":493},true,[494,498,501,504,507,510,513,516,519,522],[495],{"tag":496,"href":38,"children":497},"a",[37],[499],{"tag":496,"href":41,"children":500},[40],[502],{"tag":496,"href":44,"children":503},[43],[505],{"tag":496,"href":47,"children":506},[46],[508],{"tag":496,"href":50,"children":509},[49],[511],{"tag":496,"href":53,"children":512},[52],[514],{"tag":496,"href":56,"children":515},[55],[517],{"tag":496,"href":59,"children":518},[58],[520],{"tag":496,"href":62,"children":521},[61],[523],{"tag":496,"href":65,"children":524},[64],[526,582,626,728],{"slug":527,"published":528,"minutes":529,"category":7,"tags":530,"keywords":536,"about":547,"sources":557,"cover":576,"og":577,"expertise":68,"locales":578,"lang":70,"title":579,"description":580,"coverAlt":581},"slopsquatting-ai-package-hallucinations","2026-10-16",6,[531,532,533,534,535],"Supply chain security","Coding agents","npm","PyPI","Package hallucination",[537,538,539,540,541,542,543,544,545,546],"slopsquatting","package hallucination","ai hallucinated packages","llm package confusion attack","coding agent supply chain security","npm min-release-age","pnpm minimumReleaseAge","uv exclude-newer","npm ignore-scripts","huggingface-cli hallucinated package",[548,551,554],{"name":549,"url":550},"Typosquatting","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FTyposquatting",{"name":552,"url":553},"Supply chain attack","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FSupply_chain_attack",{"name":555,"url":556},"Hallucination (artificial intelligence)","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FHallucination_(artificial_intelligence)",[558,561,564,567,570,573],{"title":559,"url":560},"Spracklen et al.: We Have a Package for You! A Comprehensive Analysis of Package Hallucinations by Code Generating LLMs (2024)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2406.10279",{"title":562,"url":563},"Lasso Security, Bar Lanyado: Diving Deeper into AI Package Hallucinations (Mar 2024)","https:\u002F\u002Fwww.lasso.security\u002Fblog\u002Fai-package-hallucinations",{"title":565,"url":566},"npm Docs: config (min-release-age, before, ignore-scripts, allow-scripts)","https:\u002F\u002Fdocs.npmjs.com\u002Fcli\u002Fv11\u002Fusing-npm\u002Fconfig",{"title":568,"url":569},"pnpm: Dependency resolution settings (minimumReleaseAge)","https:\u002F\u002Fpnpm.io\u002Fsettings\u002Fdependency-resolution",{"title":571,"url":572},"uv: Settings reference (exclude-newer, require-hashes)","https:\u002F\u002Fdocs.astral.sh\u002Fuv\u002Freference\u002Fsettings\u002F",{"title":574,"url":575},"pip documentation: Secure installs and hash-checking mode","https:\u002F\u002Fpip.pypa.io\u002Fen\u002Fstable\u002Ftopics\u002Fsecure-installs\u002F","\u002Fimages\u002Fblog\u002Fslopsquatting-ai-package-hallucinations\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fslopsquatting-ai-package-hallucinations\u002Fog.jpg",[70,71,72],"Slopsquatting: when coding agents install packages that do not exist","How LLMs hallucinate package names, how attackers register them, and the registry, lockfile and agent-harness controls that stop the install.","Three boxes on a rail, the middle one hollow and offset, aligned squares on a tray below.",{"slug":583,"published":528,"minutes":529,"category":7,"tags":584,"keywords":589,"about":600,"sources":607,"cover":620,"og":621,"expertise":68,"locales":622,"lang":70,"title":623,"description":624,"coverAlt":625},"rag-poisoning-knowledge-base-attacks",[585,586,587,12,588],"RAG security","Data poisoning","Prompt injection","Vector database",[590,591,592,593,594,595,596,597,598,599],"rag poisoning","knowledge base poisoning","poisonedrag","rag security","indirect prompt injection rag","owasp llm08 vector and embedding weaknesses","vector database security","rag data poisoning defense","hidden text prompt injection","secure rag ingestion",[601,604,606],{"name":602,"url":603},"Retrieval-augmented generation","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FRetrieval-augmented_generation",{"name":587,"url":605},"https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FPrompt_injection",{"name":12,"url":34},[608,611,614,617],{"title":609,"url":610},"Zou et al.: PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models (2024)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2402.07867",{"title":612,"url":613},"Greshake et al.: Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection (2023)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2302.12173",{"title":615,"url":616},"OWASP Gen AI Security Project: LLM08:2025 Vector and Embedding Weaknesses","https:\u002F\u002Fgenai.owasp.org\u002Fllmrisk\u002Fllm082025-vector-and-embedding-weaknesses\u002F",{"title":618,"url":619},"OWASP Gen AI Security Project: LLM04:2025 Data and Model Poisoning","https:\u002F\u002Fgenai.owasp.org\u002Fllmrisk\u002Fllm042025-data-and-model-poisoning\u002F","\u002Fimages\u002Fblog\u002Frag-poisoning-knowledge-base-attacks\u002Fcover.webp","\u002Fimages\u002Fblog\u002Frag-poisoning-knowledge-base-attacks\u002Fog.jpg",[70,71,72],"RAG poisoning: how a few documents can steer your assistant, and how to defend the knowledge base","How attackers plant documents that RAG systems retrieve, what PoisonedRAG and OWASP show, and the ingestion, retrieval and answer controls that limit it.","Sheets feeding a container on a belt, one carrying a small block, a gate before the exit.",{"slug":627,"published":628,"minutes":629,"category":7,"tags":630,"keywords":636,"about":645,"sources":655,"cover":722,"og":723,"expertise":68,"locales":724,"lang":70,"title":725,"description":726,"coverAlt":727},"coding-agent-secrets-hygiene","2026-10-08",11,[631,632,633,634,635],"AI agents","Secrets management","Claude Code","Pre-commit scanning","CI security",[637,638,639,640,641,642,643,644],"coding agent secrets","keep secrets away from AI agents","Claude Code deny read .env","gitleaks pre-commit hook","GitHub push protection secrets","OIDC GitHub Actions short-lived credentials","rotate a leaked API key","MCP server token scope",[646,649,652],{"name":647,"url":648},"Principle of least privilege","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FPrinciple_of_least_privilege",{"name":650,"url":651},"Model Context Protocol","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FModel_Context_Protocol",{"name":653,"url":654},"Git","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FGit",[656,659,662,665,668,671,674,677,680,683,686,689,692,695,698,701,704,707,710,713,716,719],{"title":657,"url":658},"Claude Code: permissions","https:\u002F\u002Fcode.claude.com\u002Fdocs\u002Fen\u002Fpermissions",{"title":660,"url":661},"Claude Code: sandboxed Bash","https:\u002F\u002Fcode.claude.com\u002Fdocs\u002Fen\u002Fsandboxing",{"title":663,"url":664},"Claude Code: hooks","https:\u002F\u002Fcode.claude.com\u002Fdocs\u002Fen\u002Fhooks",{"title":666,"url":667},"Claude Code: data usage","https:\u002F\u002Fcode.claude.com\u002Fdocs\u002Fen\u002Fdata-usage",{"title":669,"url":670},"Claude Code: settings","https:\u002F\u002Fcode.claude.com\u002Fdocs\u002Fen\u002Fsettings",{"title":672,"url":673},"Claude Code: MCP servers","https:\u002F\u002Fcode.claude.com\u002Fdocs\u002Fen\u002Fmcp",{"title":675,"url":676},"Codex: configuration reference","https:\u002F\u002Fdevelopers.openai.com\u002Fcodex\u002Fconfig-reference",{"title":678,"url":679},"Codex: agent approvals and security","https:\u002F\u002Flearn.chatgpt.com\u002Fdocs\u002Fagent-approvals-security",{"title":681,"url":682},"Codex: advanced configuration","https:\u002F\u002Flearn.chatgpt.com\u002Fdocs\u002Fconfig-file\u002Fconfig-advanced",{"title":684,"url":685},"GitHub: about push protection","https:\u002F\u002Fdocs.github.com\u002Fen\u002Fcode-security\u002Fsecret-scanning\u002Fintroduction\u002Fabout-push-protection",{"title":687,"url":688},"GitHub: security hardening with OIDC","https:\u002F\u002Fdocs.github.com\u002Fen\u002Factions\u002Fsecurity-for-github-actions\u002Fsecurity-hardening-your-deployments\u002Fabout-security-hardening-with-openid-connect",{"title":690,"url":691},"GitHub: OpenID Connect reference","https:\u002F\u002Fdocs.github.com\u002Fen\u002Factions\u002Freference\u002Fsecurity\u002Foidc",{"title":693,"url":694},"GitHub: using secrets in Actions","https:\u002F\u002Fdocs.github.com\u002Fen\u002Factions\u002Fsecurity-guides\u002Fusing-secrets-in-github-actions",{"title":696,"url":697},"GitHub: secure use reference","https:\u002F\u002Fdocs.github.com\u002Fen\u002Factions\u002Fsecurity-for-github-actions\u002Fsecurity-guides\u002Fsecurity-hardening-for-github-actions",{"title":699,"url":700},"GitHub: removing sensitive data","https:\u002F\u002Fdocs.github.com\u002Fen\u002Fauthentication\u002Fkeeping-your-account-and-data-secure\u002Fremoving-sensitive-data-from-a-repository",{"title":702,"url":703},"gitleaks: README and latest release","https:\u002F\u002Fgithub.com\u002Fgitleaks\u002Fgitleaks",{"title":705,"url":706},"TruffleHog: README","https:\u002F\u002Fgithub.com\u002Ftrufflesecurity\u002Ftrufflehog",{"title":708,"url":709},"detect-secrets: README and latest release","https:\u002F\u002Fgithub.com\u002FYelp\u002Fdetect-secrets",{"title":711,"url":712},"Model Context Protocol: security best practices","https:\u002F\u002Fmodelcontextprotocol.io\u002Fspecification\u002F2025-06-18\u002Fbasic\u002Fsecurity_best_practices",{"title":714,"url":715},"OWASP: Secrets Management Cheat Sheet","https:\u002F\u002Fcheatsheetseries.owasp.org\u002Fcheatsheets\u002FSecrets_Management_Cheat_Sheet.html",{"title":717,"url":718},"OWASP: LLM02 sensitive information disclosure","https:\u002F\u002Fgenai.owasp.org\u002Fllmrisk\u002Fllm022025-sensitive-information-disclosure\u002F",{"title":720,"url":721},"Git: git-add documentation","https:\u002F\u002Fgit-scm.com\u002Fdocs\u002Fgit-add","\u002Fimages\u002Fblog\u002Fcoding-agent-secrets-hygiene\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fcoding-agent-secrets-hygiene\u002Fog.jpg",[70,71,72],"Coding agents and secrets: keep keys out of context, logs and commits","How secrets leak through coding agents, and the controls that stop them: deny reads, a sandbox, pre-commit scans, push protection, OIDC and rotation.","Tiny squares on a rail stop at a wall with a lock badge before an open box.",{"slug":729,"published":730,"minutes":629,"category":7,"tags":731,"keywords":737,"about":746,"sources":759,"cover":801,"og":802,"expertise":68,"locales":803,"lang":70,"title":804,"description":805,"coverAlt":806},"works-council-ai-tools-austria-germany","2026-10-02",[732,733,734,735,736],"Works council","AI coding tools","Employee monitoring","GDPR","Co-determination",[738,739,740,741,742,743,744,745],"works council AI tools","AI coding tools works council","Betriebsrat KI Mitbestimmung","§ 87 BetrVG Überwachung","§ 96 ArbVG Kontrollmaßnahmen","AI usage logs employee monitoring","works agreement for AI tools","GDPR Article 88 employee data",[747,750,753,756],{"name":748,"url":749},"General Data Protection Regulation","https:\u002F\u002Feur-lex.europa.eu\u002Feli\u002Freg\u002F2016\u002F679\u002Foj\u002Feng",{"name":751,"url":752},"EU Artificial Intelligence Act (Regulation (EU) 2024\u002F1689)","https:\u002F\u002Feur-lex.europa.eu\u002Feli\u002Freg\u002F2024\u002F1689\u002Foj\u002Feng",{"name":754,"url":755},"Arbeitsverfassungsgesetz (Austrian Labour Constitution Act)","https:\u002F\u002Fwww.ris.bka.gv.at\u002FGeltendeFassung.wxe?Abfrage=Bundesnormen&Gesetzesnummer=10008329",{"name":757,"url":758},"Betriebsverfassungsgesetz (German Works Constitution Act), § 87","https:\u002F\u002Fwww.gesetze-im-internet.de\u002Fbetrvg\u002F__87.html",[760,762,764,767,770,772,775,777,780,783,786,789,792,795,798],{"title":761,"url":755},"Arbeitsverfassungsgesetz (ArbVG), §§ 96 and 96a, consolidated text of 10 October 2026, RIS",{"title":763,"url":758},"Betriebsverfassungsgesetz (BetrVG), § 87, gesetze-im-internet.de",{"title":765,"url":766},"Betriebsverfassungsgesetz (BetrVG), § 90, gesetze-im-internet.de","https:\u002F\u002Fwww.gesetze-im-internet.de\u002Fbetrvg\u002F__90.html",{"title":768,"url":769},"Bundesdatenschutzgesetz (BDSG), § 26, gesetze-im-internet.de","https:\u002F\u002Fwww.gesetze-im-internet.de\u002Fbdsg_2018\u002F__26.html",{"title":771,"url":749},"Regulation (EU) 2016\u002F679 (GDPR), EUR-Lex",{"title":773,"url":774},"GDPR Article 5(1)(e), storage limitation, gdpr-info.eu","https:\u002F\u002Fgdpr-info.eu\u002Fart-5-gdpr\u002F",{"title":776,"url":752},"Regulation (EU) 2024\u002F1689 (AI Act), EUR-Lex",{"title":778,"url":779},"Regulation (EU) 2026\u002F1744, EUR-Lex","https:\u002F\u002Feur-lex.europa.eu\u002Feli\u002Freg\u002F2026\u002F1744\u002Foj",{"title":781,"url":782},"AI Act Article 26, AI Act Explorer","https:\u002F\u002Fartificialintelligenceact.eu\u002Farticle\u002F26\u002F",{"title":784,"url":785},"AI Act Annex III, AI Act Explorer","https:\u002F\u002Fartificialintelligenceact.eu\u002Fannex\u002F3\u002F",{"title":787,"url":788},"BAG, 1 ABR 16\u002F23 (July 2024), headset system, gesetze.co","https:\u002F\u002Fgesetze.co\u002Furteile\u002F1_ABR_16-23",{"title":790,"url":791},"ArbG Hamburg, 24 BVGa 1\u002F24: law-firm summary by CMS","https:\u002F\u002Fcms.law\u002Fde\u002Fdeu\u002Flegal-updates\u002Fkein-mitbestimmungsrecht-des-betriebsrats-bei-chatgpt-co",{"title":793,"url":794},"ArbG Hamburg, 24 BVGa 1\u002F24: law-firm summary by Gleiss Lutz","https:\u002F\u002Fwww.gleisslutz.com\u002Fde\u002Fknow-how\u002Farbeitsgericht-hamburg-zu-chatgpt-kein-mitbestimmungsrecht-des-betriebsrats",{"title":796,"url":797},"Claude Code documentation: monitoring usage","https:\u002F\u002Fcode.claude.com\u002Fdocs\u002Fen\u002Fmonitoring-usage",{"title":799,"url":800},"GitHub Docs: Copilot metrics data reference","https:\u002F\u002Fdocs.github.com\u002Fen\u002Fcopilot\u002Freference\u002Fmetrics-data","\u002Fimages\u002Fblog\u002Fworks-council-ai-tools-austria-germany\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fworks-council-ai-tools-austria-germany\u002Fog.jpg",[70,71,72],"AI coding tools and the works council: when usage logs count as monitoring","Usage logs can make an AI coding tool a monitoring system. What Austria (§ 96 ArbVG) and Germany (§ 87 BetrVG) require, and what to agree before rollout.","Two tall rectangles side by side under one wide sheet, and five small squares dropping into a box.",1792222776111]