[{"data":1,"prerenderedAt":696},["ShallowReactive",2],{"blog-rag-product-catalog-pim-data-en":3},{"slug":4,"published":5,"minutes":6,"category":7,"tags":8,"keywords":14,"about":26,"sources":36,"cover":63,"og":64,"expertise":65,"locales":66,"lang":67,"title":70,"description":71,"coverAlt":72,"metaTitle":73,"takeaways":74,"faq":80,"toc":96,"blocks":127,"others":438},"rag-product-catalog-pim-data","2026-10-17",10,"rag",[9,10,11,12,13],"RAG","PIM","Product data","B2B e-commerce","Metadata filtering",[15,16,17,18,19,20,21,22,23,24,25],"rag product catalog","rag pim data","llm product data","product catalog chatbot","akeneo rag","pimcore ai search","rag structured data","metadata filtering rag","product attributes embeddings","b2b product assistant","etim classification ai",[27,30,33],{"name":28,"url":29},"Product information management","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FProduct_information_management",{"name":31,"url":32},"Retrieval-augmented generation","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FRetrieval-augmented_generation",{"name":34,"url":35},"ETIM (standard)","https:\u002F\u002Fwww.etim-international.com\u002Fclassification\u002F",[37,40,43,46,48,51,54,57,60],{"title":38,"url":39},"Akeneo API: Products, product values, metric values and product models","https:\u002F\u002Fapi.akeneo.com\u002Fconcepts\u002Fproducts.html",{"title":41,"url":42},"Akeneo Help Center: What about products with variants?","https:\u002F\u002Fhelp.akeneo.com\u002Fserenity-take-the-power-over-your-products\u002Fserenity-what-about-products-with-variants",{"title":44,"url":45},"Pimcore documentation: Classification Store","https:\u002F\u002Fdocs.pimcore.com\u002Fplatform\u002FPimcore\u002FObjects\u002FObject_Classes\u002FData_Types\u002FClassification_Store\u002F",{"title":47,"url":35},"ETIM International: Classification",{"title":49,"url":50},"ETIM International: ETIM releases and the IXF format","https:\u002F\u002Fetim-international.com\u002F?p=754",{"title":52,"url":53},"Reddy et al.: Shopping Queries Dataset, a large-scale ESCI benchmark for improving product search (2022)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2206.06588",{"title":55,"url":56},"Amazon Science: esci-data repository","https:\u002F\u002Fgithub.com\u002Famazon-science\u002Fesci-data",{"title":58,"url":59},"Qdrant documentation: Filtering","https:\u002F\u002Fqdrant.tech\u002Fdocumentation\u002Fconcepts\u002Ffiltering\u002F",{"title":61,"url":62},"Sui et al.: Table Meets LLM, can large language models understand structured table data? (WSDM 2024)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2305.13062","\u002Fimages\u002Fblog\u002Frag-product-catalog-pim-data\u002Fcover.webp","\u002Fimages\u002Fblog\u002Frag-product-catalog-pim-data\u002Fog.jpg","b2b-ecommerce-developer",[67,68,69],"en","de","hu","RAG over product catalogues: turning PIM data into answers that hold","Product data is structured, so chunking it like PDFs fails. How to serialize PIM attributes, route spec questions to filters and keep answers exact.","A frame of nine squares with tiny symbols, funnelling down to one card.","RAG over PIM product data · Balázs Csorba",[75,76,77,78,79],"Catalogue questions come in kinds: part numbers need exact lookup, specifications need structured filters, and only descriptive questions are a good fit for vector search.","PIM values are typed and carry locale, channel and unit, as in Akeneo's locale, scope and data triples; serialize them with labels and units, never with raw attribute codes.","Index one text per product model and keep variant attributes as structured payload, so a question about a size or a thread does not depend on an embedding.","Let the LLM extract filters into a schema you validate against the attribute model, then run filtered hybrid search; prices and stock come live from the shop, not from the index.","Evaluate with expected SKUs per query and the Exact, Substitute, Complement and Irrelevant vocabulary from Amazon's ESCI dataset.",[81,84,87,90,93],{"q":82,"a":83},"Can I just embed my product export and use RAG?","You can, and it will answer descriptive questions passably. It fails on part numbers, numeric specifications and variant differences, because embeddings do not compare numbers or exact codes reliably. Combine a text index with structured filters on the attributes.",{"q":85,"a":86},"Should each variant be its own document?","Usually not for the text index. Variants of one product model share most of their text, so they crowd each other out of the top results. Index the product model, store variant attributes as payload, and resolve the exact variant with a filter or a lookup.",{"q":88,"a":89},"Where do prices and stock come from?","From the shop or ERP at answer time. In B2B, prices are often customer-specific and stock changes constantly, so an index snapshot is wrong by design. The assistant should call a price and availability endpoint with the logged-in customer's context.",{"q":91,"a":92},"Does ETIM help with RAG?","Yes, where your data is classified with it. ETIM defines classes with a uniform list of features and predefined values, which gives you consistent attribute names for filters across suppliers and a multilingual vocabulary.",{"q":94,"a":95},"How do I stop the assistant from inventing specifications?","Pass the attribute values to the model as structured data with units, instruct it to answer only from them, and make an empty attribute an explicit not specified. Test it with questions about attributes you know are missing.",[97,100,103,106,109,112,115,118,121,124],{"id":98,"title":99},"what-pim-data-looks-like","What does PIM data actually look like?",{"id":101,"title":102},"question-types","Which catalogue questions need which retrieval?",{"id":104,"title":105},"document-granularity","One document per what: product, model or variant?",{"id":107,"title":108},"serializing-attributes","How should attributes become text?",{"id":110,"title":111},"filters-not-embeddings","Why do specifications belong in filters, not embeddings?",{"id":113,"title":114},"prices-and-freshness","What should never be in the index?",{"id":116,"title":117},"multilingual-catalogues","How do locales and classifications help?",{"id":119,"title":120},"evaluating-catalogue-rag","How do you test a catalogue assistant?",{"id":122,"title":123},"where-to-start","Where should you start?",{"id":125,"title":126},"sources","Sources",[128,132,146,149,179,181,184,187,188,191,243,246,247,250,253,262,263,273,302,304,307,308,327,329,337,344,345,353,356,357,364,367,368,375,397,398,406,407],{"type":129,"content":130},"paragraph",[131],"Most RAG tutorials start from documents: PDFs, wiki pages, manuals. A product catalogue is a different animal. In a PIM the knowledge is already structured: typed attributes, units, variants, classifications, translations per locale and values per sales channel. If you flatten that into text and chunk it like a manual, you throw away exactly the structure that makes catalogue questions answerable.",{"type":129,"content":133},[134,135,140,141,145],"This post is the data side of a catalogue assistant: what PIM data looks like, how to serialize it for retrieval, which questions belong to filters instead of embeddings, and how to test that the answers name the right SKU. It builds on ",{"tag":136,"to":137,"children":138},"link","\u002Fblog\u002Fsemantic-product-search-b2b",[139],"semantic product search for B2B shops",", which covers ranking, and assumes the sync problems from ",{"tag":136,"to":142,"children":143},"\u002Fblog\u002Fpimcore-erp-delta-sync",[144],"Pimcore ERP delta sync"," are solved upstream.",{"type":147,"level":148,"id":98,"text":99},"heading",2,{"type":129,"content":150},[151,152,156,157,160,161,164,165,168,169,160,172,164,175,178],"Take Akeneo as an example. A product's ",{"tag":153,"children":154},"code",[155],"values"," are keyed by attribute code, and each code maps to an array of value objects with ",{"tag":153,"children":158},[159],"locale",", ",{"tag":153,"children":162},[163],"scope"," and ",{"tag":153,"children":166},[167],"data",". A localizable attribute has one value per locale, a scopable one has one value per channel, and an attribute that is both has one per locale and channel pair. A metric value is not a number but an object with ",{"tag":153,"children":170},[171],"amount",{"tag":153,"children":173},[174],"unit",{"tag":153,"children":176},[177],"symbol",", where the symbol comes from the measurement family.",{"type":153,"code":180},"\"values\": {\n  \"name\":        [{ \"locale\": \"de_DE\", \"scope\": null, \"data\": \"Sechskantschraube DIN 933\" }],\n  \"length\":      [{ \"locale\": null, \"scope\": null,\n                    \"data\": { \"amount\": \"50\", \"unit\": \"MILLIMETER\", \"symbol\": \"mm\" } }],\n  \"material\":    [{ \"locale\": null, \"scope\": null, \"data\": \"stainless_a2\" }],\n  \"description\": [{ \"locale\": \"de_DE\", \"scope\": \"ecommerce\", \"data\": \"...\" },\n                  { \"locale\": \"de_DE\", \"scope\": \"print\",     \"data\": \"...\" }]\n}",{"type":129,"content":182},[183],"Variants add a second layer. In Akeneo a family variant defines one or two variant levels, the axes per level and which attributes are common and which vary, for up to three levels of enrichment: a root product model, a sub product model and the variant product. Only the variant product has a SKU; the product model has a code. Pimcore takes another route with the classification store: keys with data types grouped into groups, groups attached to objects, optional localization, and quantity values that pair a number with a unit.",{"type":129,"content":185},[186],"Many technical B2B catalogues are additionally classified with ETIM, which its publisher describes as the international classification standard for technical products: classes with a uniform list of features and predefined values, available in 17 languages. ETIM 10.0 was released in December 2024 and ships as IXF, an XML format. If your suppliers deliver ETIM data, you already have a shared attribute vocabulary across brands, which is gold for filters.",{"type":147,"level":148,"id":101,"text":102},{"type":129,"content":189},[190],"The single most useful design step is to sort the questions your assistant will get by the kind of retrieval they need. For a B2B catalogue, expect a very different mix from a support wiki: many questions are lookups or constraints, and fewer are open-ended.",{"type":192,"head":193,"rows":200},"table",[194,196,198],[195],"Question",[197],"Example",[199],"Route",[201,208,215,222,229,236],[202,204,206],[203],"Identifier",[205],"Do you have 4711-050?",[207],"Exact lookup on SKU, EAN, supplier number",[209,211,213],[210],"Specification",[212],"M8 stainless screws, at least 50 mm",[214],"Structured filter, then rank",[216,218,220],[217],"Comparison",[219],"Difference between X200 and X300?",[221],"Fetch both records, compare attributes",[223,225,227],[224],"Descriptive",[226],"Something for outdoor use near salt water",[228],"Hybrid search over text",[230,232,234],[231],"Relation",[233],"Which nut fits this screw?",[235],"PIM associations, then lookup",[237,239,241],[238],"Commercial",[240],"What does it cost me, is it in stock?",[242],"Live shop or ERP call, never the index",{"type":129,"content":244},[245],"Amazon's Shopping Queries Dataset is a useful reference for the vocabulary. Its about 130,000 queries and 2.6 million human relevance judgements label each query and product pair as Exact (satisfies all query specifications), Substitute (fails some aspects but could stand in), Complement (does not fulfil the query but works with an exact item) or Irrelevant. A catalogue assistant should know which of the four it is offering, and say so: suggesting a substitute as if it were exact is the classic way to lose a B2B buyer's trust.",{"type":147,"level":148,"id":104,"text":105},{"type":129,"content":248},[249],"The first instinct is one document per SKU. For catalogues with variants this works badly in the text index. Forty lengths of the same screw share their name, description and most attributes, so a descriptive query fills the top results with near-duplicates of one product and pushes every alternative out. Embedding forty near-identical texts also costs storage and indexing time for no gain.",{"type":129,"content":251},[252],"What works better is to split the record. The text index gets one document per product model, built from the common attributes and a compact summary of the variant axes, for example lengths 20 to 120 mm, stainless A2 and A4. The variant attributes live as structured payload, either on the same point in the vector store or in a separate table keyed by model code. Retrieval finds the model by meaning, and a filter or lookup picks the variant. Simple products without variants are their own model.",{"type":254,"attrs":255,"inner":259,"caption":260},"diagram",{"viewBox":256,"role":257,"aria-labelledby":258},"0 0 720 170","img","pim1-t pim1-d","\u003Ctitle id=\"pim1-t\">Splitting a PIM record for retrieval\u003C\u002Ftitle>\u003Cdesc id=\"pim1-d\">A PIM export is serialized per product model and locale into text for the hybrid index, while variant attributes with units go into a structured store. At question time a parser extracts filters, filtered hybrid search finds candidates, and the answer cites SKUs with live prices.\u003C\u002Fdesc>\u003Cdefs>\u003Cmarker id=\"pim1-ah\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto-start-reverse\">\u003Cpath d=\"M0 0L10 5L0 10z\" class=\"d-head\" \u002F>\u003C\u002Fmarker>\u003C\u002Fdefs>\u003Crect x=\"20\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-box\" \u002F>\u003Ctext x=\"79\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">PIM export\u003C\u002Ftext>\u003Ctext x=\"79\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">values, variants\u003C\u002Ftext>\u003Cpath d=\"M138 85H158\" class=\"d-line\" marker-end=\"url(#pim1-ah)\" \u002F>\u003Crect x=\"160\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-accent\" \u002F>\u003Ctext x=\"220\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Serializer\u003C\u002Ftext>\u003Ctext x=\"220\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">labels + units\u003C\u002Ftext>\u003Cpath d=\"M279 85H299\" class=\"d-line\" marker-end=\"url(#pim1-ah)\" \u002F>\u003Crect x=\"301\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-sky\" \u002F>\u003Ctext x=\"360\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Text index\u003C\u002Ftext>\u003Ctext x=\"360\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">one per model\u003C\u002Ftext>\u003Cpath d=\"M419 85H439\" class=\"d-line\" marker-end=\"url(#pim1-ah)\" \u002F>\u003Crect x=\"441\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-gold\" \u002F>\u003Ctext x=\"500\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Payload\u003C\u002Ftext>\u003Ctext x=\"500\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">typed variants\u003C\u002Ftext>\u003Cpath d=\"M560 85H580\" class=\"d-line\" marker-end=\"url(#pim1-ah)\" \u002F>\u003Crect x=\"582\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-mint\" \u002F>\u003Ctext x=\"641\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Answer\u003C\u002Ftext>\u003Ctext x=\"641\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">SKU + live price\u003C\u002Ftext>",[261],"Text answers what a product is; structure answers which variant it is.",{"type":147,"level":148,"id":107,"text":108},{"type":129,"content":264},[265,266,164,269,272],"The serializer is the most underrated piece of a catalogue RAG pipeline. Embedding models and LLMs read labels, not codes: ",{"tag":153,"children":267},[268],"stainless_a2",{"tag":153,"children":270},[271],"MILLIMETER"," mean little, Material: stainless steel A2 and Length: 50 mm mean a lot. A few rules I apply:",{"type":274,"items":275},"list",[276,282,287,292,297],[277,281],{"tag":278,"children":279},"strong",[280],"Resolve every code to its label"," in the target locale: attribute names, select options, units via their symbol.",[283,286],{"tag":278,"children":284},[285],"Pick one channel."," Scopable values differ per channel; index the e-commerce scope, not the print catalogue text.",[288,291],{"tag":278,"children":289},[290],"One text per locale,"," with an explicit fallback rule, so a German query never retrieves a half-English record by accident.",[293,296],{"tag":278,"children":294},[295],"Drop empty and internal attributes:"," cost prices, workflow flags, import timestamps. They add noise and sometimes leak.",[298,301],{"tag":278,"children":299},[300],"Put identifiers in the text too:"," SKU, EAN and supplier numbers belong in the BM25 field, because users paste them.",{"type":153,"code":303},"def serialize(model, variants, locale, labels, channel=\"ecommerce\"):\n    def val(values, code):\n        for v in values.get(code, []):\n            if v[\"locale\"] in (locale, None) and v[\"scope\"] in (channel, None):\n                d = v[\"data\"]\n                if isinstance(d, dict) and \"amount\" in d:      # metric value\n                    return f'{d[\"amount\"]} {d[\"symbol\"]}'\n                return labels.option(code, d, locale) if isinstance(d, str) else d\n        return None\n\n    lines = [f'# {val(model[\"values\"], \"name\")}']\n    for code in labels.common_attributes(model[\"family\"]):\n        v = val(model[\"values\"], code)\n        if v not in (None, \"\", []):\n            lines.append(f'{labels.attribute(code, locale)}: {v}')\n    # summarise the variant axes instead of repeating every variant\n    for axis in labels.axes(model[\"family_variant\"]):\n        seen = sorted({str(val(x[\"values\"], axis)) for x in variants})\n        lines.append(f'{labels.attribute(axis, locale)} (variants): {\", \".join(seen)}')\n    lines.append(\"SKUs: \" + \" \".join(x[\"identifier\"] for x in variants))\n    return \"\\n\".join(lines)",{"type":129,"content":305},[306],"Whether you render the attributes as key-value lines, a Markdown table or JSON is worth a test rather than a belief. The Table Meets LLM study found that LLM performance on table tasks varied with input choices such as format and content order; your embedding model and your queries decide which format retrieves best. I start with label: value lines because they work for BM25, embeddings and the LLM alike.",{"type":147,"level":148,"id":110,"text":111},{"type":129,"content":309},[310,311,314,315,318,319,322,323,326],"A dense embedding compresses a text into a direction that captures what it is about. It is not built to tell 48 mm from 50 mm, or to know that at least 50 excludes 48. BM25 matches the token 50 but has no idea of ranges or units either. So constraints should be extracted from the question and applied as filters on typed fields. Vector databases support this directly; in Qdrant a ",{"tag":153,"children":312},[313],"must"," clause with a ",{"tag":153,"children":316},[317],"range"," condition and ",{"tag":153,"children":320},[321],"gte"," or ",{"tag":153,"children":324},[325],"lte"," bounds does it, and the docs recommend payload indexes on every field you filter on.",{"type":153,"code":328},"{\n  \"query\": \"\u003Cembedding of: stainless hex screw for outdoor use>\",\n  \"filter\": {\n    \"must\": [\n      { \"key\": \"thread\",        \"match\": { \"value\": \"M8\" } },\n      { \"key\": \"material\",      \"match\": { \"any\": [\"stainless_a2\", \"stainless_a4\"] } },\n      { \"key\": \"length_mm\",     \"range\": { \"gte\": 50 } },\n      { \"key\": \"locale\",        \"match\": { \"value\": \"de_DE\" } }\n    ]\n  },\n  \"limit\": 20\n}",{"type":129,"content":330},[331,332,336],"The filter extraction is a small LLM call with structured output: give the model the list of filterable attributes with their types, allowed options and canonical units, and have it return a JSON object. Then validate it in code. Unknown attribute codes are dropped, option values must exist, and units are normalised before comparison, so 5 cm becomes 50 mm. This is the same discipline as in ",{"tag":136,"to":333,"children":334},"\u002Fblog\u002Ftext-to-sql-llm-guardrails",[335],"text-to-SQL with guardrails",": the model proposes, a schema decides. If a constraint cannot be mapped, keep it in the text query rather than silently dropping it, and tell the user which filters were applied.",{"type":338,"variant":339,"title":340,"body":341},"callout","warn","An empty attribute is not a no",[342],[343],"Catalogue data is incomplete, often badly so for long-tail articles. If the assistant gets a record without a corrosion class and is asked whether the screw is suitable for salt water, the honest answer is that the data does not say. Pass missing values explicitly as not specified, instruct the model never to infer a technical value from the product name or from similar products, and test it with questions about attributes you know are empty. A wrong load rating in a B2B context is a liability question, not a UX glitch.",{"type":147,"level":148,"id":113,"text":114},{"type":129,"content":346},[347,348,352],"Prices, stock and delivery dates. In B2B shops prices are frequently customer-specific, with contract prices, scales and discounts, and stock moves every minute. An index snapshot of those values is wrong by design and, worse, may show one customer's conditions to another. Keep them out of the text and the payload. When the answer needs them, the assistant calls the shop's price and availability endpoint with the logged-in customer's session, the same way the storefront does. I covered the tool side of that in ",{"tag":136,"to":349,"children":350},"\u002Fblog\u002Fai-assistant-b2b-shop-customer-support",[351],"an AI assistant for B2B shop support",".",{"type":129,"content":354},[355],"For the rest, freshness is a sync problem. Re-serialize and re-embed a product model when any of its common attributes or labels change, and update only payload when a variant attribute changes, which is cheap because no embedding is involved. A changed option label, say a material renamed in German, touches every model that uses it, so key your change detection on the serialized text hash, not on the PIM's updated timestamp alone.",{"type":147,"level":148,"id":116,"text":117},{"type":129,"content":358},[359,360,352],"Catalogue data is often better translated than any wiki, because the PIM forces translations per attribute. Use that: one serialized text per locale, filtered by locale at query time, with a documented fallback when a translation is missing. Options and units are language-neutral in the payload, so filters work identically in German, English and Hungarian, and only the text index is per language. The general trade-offs of cross-lingual retrieval are in ",{"tag":136,"to":361,"children":362},"\u002Fblog\u002Fmultilingual-rag-german-hungarian",[363],"multilingual RAG for German and Hungarian",{"type":129,"content":365},[366],"Classifications such as ETIM add a second benefit. Because a class has a uniform feature list, the filter extraction prompt can be built per class: once the question is mapped to, say, a screw class, only that class's features are offered as filters. That keeps the prompt short and the extracted filters valid, even across suppliers whose own attribute names differ.",{"type":147,"level":148,"id":119,"text":120},{"type":129,"content":369},[370,371,352],"Catalogue RAG has an advantage over document RAG: the ground truth is crisp. For each test query you can write down the SKUs or product models that are correct, which turns retrieval evaluation into hit rate and recall at k on IDs instead of judging passages. I build the set from search logs and sales questions, grouped by the question types in the table above, and label the expected results with the ESCI categories so a substitute is not counted as a hit. The metrics themselves are covered in ",{"tag":136,"to":372,"children":373},"\u002Fblog\u002Frag-evaluation-metrics",[374],"evaluating RAG",{"type":274,"items":376},[377,382,387,392],[378,381],{"tag":278,"children":379},[380],"Identifier queries:"," expect 100% exact hits; anything less is a tokenizer or field-mapping bug, not a model problem.",[383,386],{"tag":278,"children":384},[385],"Specification queries:"," check the extracted filter against a hand-written expected filter before looking at results.",[388,391],{"tag":278,"children":389},[390],"Missing-data queries:"," the correct answer is not specified; count invented values as failures.",[393,396],{"tag":278,"children":394},[395],"Commercial queries:"," verify that the price comes from the live call, by testing with two customers who have different conditions.",{"type":147,"level":148,"id":122,"text":123},{"type":129,"content":399},[400,401,405],"Start with the serializer and the identifier path, because they are cheap and remove the most embarrassing failures. Then add filter extraction for the five to ten attributes that appear most in real questions, not for all of them. Only after that tune the text index, chunk sizes and rerankers. If your catalogue is also published with structured data, the same attribute mapping can feed both, which I described in ",{"tag":136,"to":402,"children":403},"\u002Fblog\u002Fschema-org-structured-data-b2b-shop",[404],"Schema.org structured data for B2B shops",". The pattern underneath is simple: the PIM already did the hard work of structuring product knowledge, and a good catalogue assistant keeps that structure all the way to the answer.",{"type":147,"level":148,"id":125,"text":126},{"type":274,"ordered":408,"items":409},true,[410,414,417,420,423,426,429,432,435],[411],{"tag":412,"href":39,"children":413},"a",[38],[415],{"tag":412,"href":42,"children":416},[41],[418],{"tag":412,"href":45,"children":419},[44],[421],{"tag":412,"href":35,"children":422},[47],[424],{"tag":412,"href":50,"children":425},[49],[427],{"tag":412,"href":53,"children":428},[52],[430],{"tag":412,"href":56,"children":431},[55],[433],{"tag":412,"href":59,"children":434},[58],[436],{"tag":412,"href":62,"children":437},[61],[439,514,571,629],{"slug":440,"published":5,"minutes":441,"category":7,"tags":442,"keywords":447,"about":460,"sources":470,"cover":507,"og":508,"expertise":509,"locales":510,"lang":67,"title":511,"description":512,"coverAlt":513},"embedding-model-selection-2026",9,[443,9,444,445,446],"Embeddings","MTEB","Vector search","Model selection",[448,449,450,451,452,453,454,455,456,457,458,459],"best embedding model 2026","how to choose an embedding model","embedding model comparison","MTEB leaderboard explained","RTEB benchmark","open source embedding model","multilingual embedding model","voyage-4 vs openai embeddings","Qwen3-Embedding","EmbeddingGemma","embedding model for RAG","switching embedding models re-embed",[461,464,467],{"name":462,"url":463},"Word embedding","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FWord_embedding",{"name":465,"url":466},"Information retrieval","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FInformation_retrieval",{"name":468,"url":469},"Discounted cumulative gain","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FDiscounted_cumulative_gain",[471,474,477,480,483,486,489,492,495,498,501,504],{"title":472,"url":473},"Enevoldsen et al.: MMTEB, Massive Multilingual Text Embedding Benchmark (2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2502.13595",{"title":475,"url":476},"Hugging Face: Introducing RTEB, a new standard for retrieval evaluation (Oct 2025)","https:\u002F\u002Fhuggingface.co\u002Fblog\u002Frteb",{"title":478,"url":479},"OpenAI: Vector embeddings guide","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fguides\u002Fembeddings",{"title":481,"url":482},"OpenAI: API pricing","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fpricing",{"title":484,"url":485},"Voyage AI: The Voyage 4 model family, shared embedding space with MoE architecture (Jan 2026)","https:\u002F\u002Fblog.voyageai.com\u002F2026\u002F01\u002F15\u002Fvoyage-4\u002F",{"title":487,"url":488},"Voyage AI: Text embeddings documentation","https:\u002F\u002Fdocs.voyageai.com\u002Fdocs\u002Fembeddings",{"title":490,"url":491},"Voyage AI: Pricing","https:\u002F\u002Fdocs.voyageai.com\u002Fdocs\u002Fpricing",{"title":493,"url":494},"Google: Gemini API embeddings documentation","https:\u002F\u002Fai.google.dev\u002Fgemini-api\u002Fdocs\u002Fembeddings",{"title":496,"url":497},"Google: Gemini API pricing","https:\u002F\u002Fai.google.dev\u002Fgemini-api\u002Fdocs\u002Fpricing",{"title":499,"url":500},"Cohere: Embed models documentation","https:\u002F\u002Fdocs.cohere.com\u002Fdocs\u002Fcohere-embed",{"title":502,"url":503},"Qwen: Qwen3-Embedding-8B model card","https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen3-Embedding-8B",{"title":505,"url":506},"Google Developers Blog: Introducing EmbeddingGemma (Sep 2025)","https:\u002F\u002Fdevelopers.googleblog.com\u002Fen\u002Fintroducing-embeddinggemma\u002F","\u002Fimages\u002Fblog\u002Fembedding-model-selection-2026\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fembedding-model-selection-2026\u002Fog.jpg","ai-engineer",[67,68,69],"Choosing an embedding model in 2026: leaderboards, costs and a bake-off on your own data","How to pick an embedding model for RAG in 2026: what MTEB and RTEB tell you, hosted vs open weights, prices, dimensions, and a one-week bake-off.","Three columns of stacked blocks, the middle tallest, each measured by a short bar and a wide bracket.",{"slug":515,"published":5,"minutes":6,"category":7,"tags":516,"keywords":520,"about":532,"sources":540,"cover":565,"og":566,"expertise":509,"locales":567,"lang":67,"title":568,"description":569,"coverAlt":570},"contextual-retrieval-late-chunking",[9,517,443,518,519],"Chunking","Contextual retrieval","Late chunking",[521,522,523,524,525,526,527,528,529,530,531],"contextual retrieval","anthropic contextual retrieval","late chunking","contextual chunk embeddings","contextual embeddings rag","contextual bm25","chunk context rag","voyage-context-3","voyage-context-4","late chunking vs contextual retrieval","rag chunking context loss",[533,534,537],{"name":31,"url":32},{"name":535,"url":536},"Okapi BM25","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FOkapi_BM25",{"name":538,"url":539},"Sentence embedding","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FSentence_embedding",[541,544,547,550,553,556,559,562],{"title":542,"url":543},"Anthropic: Introducing Contextual Retrieval (Sep 2024)","https:\u002F\u002Fwww.anthropic.com\u002Fnews\u002Fcontextual-retrieval",{"title":545,"url":546},"Günther et al.: Late Chunking, Contextual Chunk Embeddings Using Long-Context Embedding Models (2024, rev. 2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2409.04701",{"title":548,"url":549},"Jina AI: Late Chunking in Long-Context Embedding Models (Aug 2024)","https:\u002F\u002Fjina.ai\u002Fnews\u002Flate-chunking-in-long-context-embedding-models\u002F",{"title":551,"url":552},"Voyage AI: Introducing voyage-context-3 (Jul 2025)","https:\u002F\u002Fblog.voyageai.com\u002F2025\u002F07\u002F23\u002Fvoyage-context-3\u002F",{"title":554,"url":555},"Voyage AI: voyage-context-4 (Jun 2026)","https:\u002F\u002Fblog.voyageai.com\u002F2026\u002F06\u002F29\u002Fvoyage-context-4\u002F",{"title":557,"url":558},"Voyage AI: Contextualized chunk embeddings documentation","https:\u002F\u002Fdocs.voyageai.com\u002Fdocs\u002Fcontextualized-chunk-embeddings",{"title":560,"url":561},"Anthropic: Claude API pricing","https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fabout-claude\u002Fpricing",{"title":563,"url":564},"Qu, Tu and Bao: Is Semantic Chunking Worth the Computational Cost? (2024)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.13070","\u002Fimages\u002Fblog\u002Fcontextual-retrieval-late-chunking\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fcontextual-retrieval-late-chunking\u002Fog.jpg",[67,68,69],"Contextual retrieval and late chunking: giving RAG chunks back their context","Chunks lose their document. How contextual retrieval, late chunking and contextualized embedding models fix that, what each costs, and when to skip it.","Four loose square pieces gathered by a wide bracket below, each joined upward to a large sheet.",{"slug":572,"published":5,"minutes":441,"category":7,"tags":573,"keywords":577,"about":590,"sources":598,"cover":623,"og":624,"expertise":509,"locales":625,"lang":67,"title":626,"description":627,"coverAlt":628},"rag-access-control-permissions",[9,574,575,576,445],"Access control","Security","Multi-tenancy",[578,579,580,581,582,583,584,585,586,587,588,589],"rag access control","permission aware rag","document level security rag","rag multi tenancy","vector database access control","pgvector row level security","security trimming azure ai search","openfga rag authorization","qdrant multitenancy","pinecone namespaces multitenancy","owasp llm08 vector and embedding weaknesses","rag data leakage",[591,594,597],{"name":592,"url":593},"Access-control list","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FAccess-control_list",{"name":595,"url":596},"Multitenancy","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FMultitenancy",{"name":31,"url":32},[599,602,605,608,611,614,617,620],{"title":600,"url":601},"OWASP GenAI Security Project: LLM08:2025 Vector and Embedding Weaknesses","https:\u002F\u002Fgenai.owasp.org\u002Fllmrisk\u002Fllm082025-vector-and-embedding-weaknesses\u002F",{"title":603,"url":604},"PostgreSQL documentation: Row Security Policies","https:\u002F\u002Fwww.postgresql.org\u002Fdocs\u002Fcurrent\u002Fddl-rowsecurity.html",{"title":606,"url":607},"pgvector README: filtering and iterative index scans","https:\u002F\u002Fgithub.com\u002Fpgvector\u002Fpgvector",{"title":609,"url":610},"Qdrant: Multitenancy and partitions guide","https:\u002F\u002Fqdrant.tech\u002Fdocumentation\u002Fguides\u002Fmultiple-partitions\u002F",{"title":612,"url":613},"Pinecone: Implement multitenancy using namespaces","https:\u002F\u002Fdocs.pinecone.io\u002Fguides\u002Findex-data\u002Fimplement-multitenancy",{"title":615,"url":616},"Microsoft Learn: Document-level access control in Azure AI Search","https:\u002F\u002Flearn.microsoft.com\u002Fen-us\u002Fazure\u002Fsearch\u002Fsearch-document-level-access-overview",{"title":618,"url":619},"OpenFGA: RAG authorization use case","https:\u002F\u002Fopenfga.dev\u002Fdocs\u002Fuse-cases\u002Frag-authorization",{"title":621,"url":622},"Morris et al.: Text Embeddings Reveal (Almost) As Much As Text (EMNLP 2023)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2310.06816","\u002Fimages\u002Fblog\u002Frag-access-control-permissions\u002Fcover.webp","\u002Fimages\u002Fblog\u002Frag-access-control-permissions\u002Fog.jpg",[67,68,69],"Access control for RAG: permission-aware retrieval without leaking documents","How RAG assistants leak documents and how to stop it: ACLs on every chunk, pre-filtering, row-level security in pgvector, permission sync and leak tests.","Seven cards in a row, six with a lock badge, one bare card sending an arrow outward.",{"slug":630,"published":5,"minutes":631,"category":7,"tags":632,"keywords":637,"about":648,"sources":656,"cover":690,"og":691,"expertise":509,"locales":692,"lang":67,"title":693,"description":694,"coverAlt":695},"conversational-search-query-rewriting",12,[633,634,9,635,636],"Conversational search","Query rewriting","HyDE","E-commerce",[638,639,640,641,642,643,644,645,646,647],"conversational search query rewriting","coreference resolution search query","HyDE hypothetical document embeddings","multi-query expansion RAG","LLM filter extraction","B2B product search assistant","conversational product search","query rewriting latency","recall@k evaluation","standalone query generation",[649,652,655],{"name":650,"url":651},"Query expansion","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FQuery_expansion",{"name":653,"url":654},"Coreference","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FCoreference",{"name":535,"url":536},[657,660,663,666,669,672,675,678,681,684,687],{"title":658,"url":659},"Gao et al.: Precise Zero-Shot Dense Retrieval without Relevance Labels, ACL 2023","https:\u002F\u002Faclanthology.org\u002F2023.acl-long.99\u002F",{"title":661,"url":662},"UTokyo-Hitotsubashi at TREC 2025 RAG Track: HyDE-Enhanced Sparse-Dense Retrieval Fusion with LLM Reranking","https:\u002F\u002Ftrec.nist.gov\u002Fpubs\u002Ftrec34\u002Fpapers\u002FUTokyo.rag.pdf",{"title":664,"url":665},"Hypothetical Documents or Knowledge Leakage? Rethinking LLM-based Query Expansion","https:\u002F\u002Faclanthology.org\u002F2025.findings-acl.980.pdf",{"title":667,"url":668},"ConvSearch-R1: Enhancing Query Reformulation for Conversational Search with Reasoning via Reinforcement Learning","https:\u002F\u002Faclanthology.org\u002F2025.emnlp-main.1349\u002F",{"title":670,"url":671},"CFDA and CLIP at TREC iKAT 2025: Query Reformulation and Rank Fusion","https:\u002F\u002Farxiv.org\u002Fabs\u002F2509.15588",{"title":673,"url":674},"MiniELM: A Lightweight and Adaptive Query Rewriting Framework for E-Commerce Search Optimization","https:\u002F\u002Faclanthology.org\u002F2025.findings-acl.363\u002F",{"title":676,"url":677},"Microsoft Learn: Rewrite queries with semantic ranker in Azure AI Search (preview)","https:\u002F\u002Flearn.microsoft.com\u002Fen-us\u002Fazure\u002Fsearch\u002Fsemantic-how-to-query-rewrite",{"title":679,"url":680},"Microsoft Learn: Agentic retrieval overview in Azure AI Search","https:\u002F\u002Flearn.microsoft.com\u002Fen-us\u002Fazure\u002Fsearch\u002Fagentic-retrieval-overview",{"title":682,"url":683},"Microsoft Tech Community: Generative query rewriting and a new semantic ranker","https:\u002F\u002Ftechcommunity.microsoft.com\u002Fblog\u002Fazure-ai-services-blog\u002Fraising-the-bar-for-rag-excellence-query-rewriting-and-new-semantic-ranker\u002F4302729\u002F",{"title":685,"url":686},"Amazon Bedrock User Guide: Configure and customize queries and response generation","https:\u002F\u002Fdocs.aws.amazon.com\u002Fbedrock\u002Flatest\u002Fuserguide\u002Fkb-test-config.html",{"title":688,"url":689},"Ragas documentation: List of available metrics","https:\u002F\u002Fdocs.ragas.io\u002Fen\u002Fstable\u002Fconcepts\u002Fmetrics\u002Favailable_metrics\u002F","\u002Fimages\u002Fblog\u002Fconversational-search-query-rewriting\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fconversational-search-query-rewriting\u002Fog.jpg",[67,68,69],"Query rewriting for conversational search: context, HyDE and filters","How to rewrite follow-up questions in a B2B shop assistant: coreference, HyDE, multi-query expansion, filter extraction, latency and evaluation.","A speech bubble, then a rewritten card, then a funnel over results, linked by arrows.",1792245345483]