RAG over product catalogues: turning PIM data into answers that hold

Most RAG tutorials start from documents: PDFs, wiki pages, manuals. A product catalogue is a different animal. In a PIM the knowledge is already structured: typed attributes, units, variants, classifications, translations per locale and values per sales channel. If you flatten that into text and chunk it like a manual, you throw away exactly the structure that makes catalogue questions answerable.

This post is the data side of a catalogue assistant: what PIM data looks like, how to serialize it for retrieval, which questions belong to filters instead of embeddings, and how to test that the answers name the right SKU. It builds on semantic product search for B2B shops, which covers ranking, and assumes the sync problems from Pimcore ERP delta sync are solved upstream.

What does PIM data actually look like?

Take Akeneo as an example. A product's values are keyed by attribute code, and each code maps to an array of value objects with locale, scope and data. A localizable attribute has one value per locale, a scopable one has one value per channel, and an attribute that is both has one per locale and channel pair. A metric value is not a number but an object with amount, unit and symbol, where the symbol comes from the measurement family.

"values": {
  "name":        [{ "locale": "de_DE", "scope": null, "data": "Sechskantschraube DIN 933" }],
  "length":      [{ "locale": null, "scope": null,
                    "data": { "amount": "50", "unit": "MILLIMETER", "symbol": "mm" } }],
  "material":    [{ "locale": null, "scope": null, "data": "stainless_a2" }],
  "description": [{ "locale": "de_DE", "scope": "ecommerce", "data": "..." },
                  { "locale": "de_DE", "scope": "print",     "data": "..." }]
}

Variants add a second layer. In Akeneo a family variant defines one or two variant levels, the axes per level and which attributes are common and which vary, for up to three levels of enrichment: a root product model, a sub product model and the variant product. Only the variant product has a SKU; the product model has a code. Pimcore takes another route with the classification store: keys with data types grouped into groups, groups attached to objects, optional localization, and quantity values that pair a number with a unit.

Many technical B2B catalogues are additionally classified with ETIM, which its publisher describes as the international classification standard for technical products: classes with a uniform list of features and predefined values, available in 17 languages. ETIM 10.0 was released in December 2024 and ships as IXF, an XML format. If your suppliers deliver ETIM data, you already have a shared attribute vocabulary across brands, which is gold for filters.

Which catalogue questions need which retrieval?

The single most useful design step is to sort the questions your assistant will get by the kind of retrieval they need. For a B2B catalogue, expect a very different mix from a support wiki: many questions are lookups or constraints, and fewer are open-ended.

Amazon's Shopping Queries Dataset is a useful reference for the vocabulary. Its about 130,000 queries and 2.6 million human relevance judgements label each query and product pair as Exact (satisfies all query specifications), Substitute (fails some aspects but could stand in), Complement (does not fulfil the query but works with an exact item) or Irrelevant. A catalogue assistant should know which of the four it is offering, and say so: suggesting a substitute as if it were exact is the classic way to lose a B2B buyer's trust.

One document per what: product, model or variant?

The first instinct is one document per SKU. For catalogues with variants this works badly in the text index. Forty lengths of the same screw share their name, description and most attributes, so a descriptive query fills the top results with near-duplicates of one product and pushes every alternative out. Embedding forty near-identical texts also costs storage and indexing time for no gain.

What works better is to split the record. The text index gets one document per product model, built from the common attributes and a compact summary of the variant axes, for example lengths 20 to 120 mm, stainless A2 and A4. The variant attributes live as structured payload, either on the same point in the vector store or in a separate table keyed by model code. Retrieval finds the model by meaning, and a filter or lookup picks the variant. Simple products without variants are their own model.

How should attributes become text?

The serializer is the most underrated piece of a catalogue RAG pipeline. Embedding models and LLMs read labels, not codes: stainless_a2 and MILLIMETER mean little, Material: stainless steel A2 and Length: 50 mm mean a lot. A few rules I apply:

• Resolve every code to its label in the target locale: attribute names, select options, units via their symbol.
• Pick one channel. Scopable values differ per channel; index the e-commerce scope, not the print catalogue text.
• One text per locale, with an explicit fallback rule, so a German query never retrieves a half-English record by accident.
• Drop empty and internal attributes: cost prices, workflow flags, import timestamps. They add noise and sometimes leak.
• Put identifiers in the text too: SKU, EAN and supplier numbers belong in the BM25 field, because users paste them.

def serialize(model, variants, locale, labels, channel="ecommerce"):
    def val(values, code):
        for v in values.get(code, []):
            if v["locale"] in (locale, None) and v["scope"] in (channel, None):
                d = v["data"]
                if isinstance(d, dict) and "amount" in d:      # metric value
                    return f'{d["amount"]} {d["symbol"]}'
                return labels.option(code, d, locale) if isinstance(d, str) else d
        return None

    lines = [f'# {val(model["values"], "name")}']
    for code in labels.common_attributes(model["family"]):
        v = val(model["values"], code)
        if v not in (None, "", []):
            lines.append(f'{labels.attribute(code, locale)}: {v}')
    # summarise the variant axes instead of repeating every variant
    for axis in labels.axes(model["family_variant"]):
        seen = sorted({str(val(x["values"], axis)) for x in variants})
        lines.append(f'{labels.attribute(axis, locale)} (variants): {", ".join(seen)}')
    lines.append("SKUs: " + " ".join(x["identifier"] for x in variants))
    return "\n".join(lines)

Whether you render the attributes as key-value lines, a Markdown table or JSON is worth a test rather than a belief. The Table Meets LLM study found that LLM performance on table tasks varied with input choices such as format and content order; your embedding model and your queries decide which format retrieves best. I start with label: value lines because they work for BM25, embeddings and the LLM alike.

Why do specifications belong in filters, not embeddings?

A dense embedding compresses a text into a direction that captures what it is about. It is not built to tell 48 mm from 50 mm, or to know that at least 50 excludes 48. BM25 matches the token 50 but has no idea of ranges or units either. So constraints should be extracted from the question and applied as filters on typed fields. Vector databases support this directly; in Qdrant a must clause with a range condition and gte or lte bounds does it, and the docs recommend payload indexes on every field you filter on.

{
  "query": "<embedding of: stainless hex screw for outdoor use>",
  "filter": {
    "must": [
      { "key": "thread",        "match": { "value": "M8" } },
      { "key": "material",      "match": { "any": ["stainless_a2", "stainless_a4"] } },
      { "key": "length_mm",     "range": { "gte": 50 } },
      { "key": "locale",        "match": { "value": "de_DE" } }
    ]
  },
  "limit": 20
}

The filter extraction is a small LLM call with structured output: give the model the list of filterable attributes with their types, allowed options and canonical units, and have it return a JSON object. Then validate it in code. Unknown attribute codes are dropped, option values must exist, and units are normalised before comparison, so 5 cm becomes 50 mm. This is the same discipline as in text-to-SQL with guardrails: the model proposes, a schema decides. If a constraint cannot be mapped, keep it in the text query rather than silently dropping it, and tell the user which filters were applied.

What should never be in the index?

Prices, stock and delivery dates. In B2B shops prices are frequently customer-specific, with contract prices, scales and discounts, and stock moves every minute. An index snapshot of those values is wrong by design and, worse, may show one customer's conditions to another. Keep them out of the text and the payload. When the answer needs them, the assistant calls the shop's price and availability endpoint with the logged-in customer's session, the same way the storefront does. I covered the tool side of that in an AI assistant for B2B shop support.

For the rest, freshness is a sync problem. Re-serialize and re-embed a product model when any of its common attributes or labels change, and update only payload when a variant attribute changes, which is cheap because no embedding is involved. A changed option label, say a material renamed in German, touches every model that uses it, so key your change detection on the serialized text hash, not on the PIM's updated timestamp alone.

How do locales and classifications help?

Catalogue data is often better translated than any wiki, because the PIM forces translations per attribute. Use that: one serialized text per locale, filtered by locale at query time, with a documented fallback when a translation is missing. Options and units are language-neutral in the payload, so filters work identically in German, English and Hungarian, and only the text index is per language. The general trade-offs of cross-lingual retrieval are in multilingual RAG for German and Hungarian.

Classifications such as ETIM add a second benefit. Because a class has a uniform feature list, the filter extraction prompt can be built per class: once the question is mapped to, say, a screw class, only that class's features are offered as filters. That keeps the prompt short and the extracted filters valid, even across suppliers whose own attribute names differ.

How do you test a catalogue assistant?

Catalogue RAG has an advantage over document RAG: the ground truth is crisp. For each test query you can write down the SKUs or product models that are correct, which turns retrieval evaluation into hit rate and recall at k on IDs instead of judging passages. I build the set from search logs and sales questions, grouped by the question types in the table above, and label the expected results with the ESCI categories so a substitute is not counted as a hit. The metrics themselves are covered in evaluating RAG.

• Identifier queries: expect 100% exact hits; anything less is a tokenizer or field-mapping bug, not a model problem.
• Specification queries: check the extracted filter against a hand-written expected filter before looking at results.
• Missing-data queries: the correct answer is not specified; count invented values as failures.
• Commercial queries: verify that the price comes from the live call, by testing with two customers who have different conditions.

Where should you start?

Start with the serializer and the identifier path, because they are cheap and remove the most embarrassing failures. Then add filter extraction for the five to ten attributes that appear most in real questions, not for all of them. Only after that tune the text index, chunk sizes and rerankers. If your catalogue is also published with structured data, the same attribute mapping can feed both, which I described in Schema.org structured data for B2B shops. The pattern underneath is simple: the PIM already did the hard work of structuring product knowledge, and a good catalogue assistant keeps that structure all the way to the answer.

Sources

1. Akeneo API: Products, product values, metric values and product models
2. Akeneo Help Center: What about products with variants?
3. Pimcore documentation: Classification Store
4. ETIM International: Classification
5. ETIM International: ETIM releases and the IXF format
6. Reddy et al.: Shopping Queries Dataset, a large-scale ESCI benchmark for improving product search (2022)
7. Amazon Science: esci-data repository
8. Qdrant documentation: Filtering
9. Sui et al.: Table Meets LLM, can large language models understand structured table data? (WSDM 2024)