Multilingual RAG for German and Hungarian: embeddings, compounds and cross-lingual search

Most RAG tutorials assume English documents and English questions. Projects in Austria, Germany and Hungary rarely look like that: product data in German, support tickets in Hungarian, manuals in English, and users who ask in whichever language they think in, often without umlauts or accents. Retrieval that works in a demo quietly drops half the relevant passages.

This article covers the decisions that matter for German and Hungarian content: index strategy, multilingual embedding models, what German compounds and Hungarian suffixes do to keyword search, accent folding, language metadata, the answer language, and how to evaluate per language. The same patterns carry over to other morphologically rich languages.

Why is multilingual RAG harder than English RAG?

Three problems stack up. First, morphology: German glues nouns into compounds (Kaffeetasse, Druckluftschlauch), and Hungarian is agglutinative, so one stem appears with many suffixes (ház, házban, házainkban: house, in the house, in our houses). Exact-match search misses most of those forms. Second, cross-lingual queries: a German question whose answer sits in an English manual finds nothing with keyword search. Third, mixed corpora: the same document in three languages can crowd the top results with near-duplicates.

Which index strategy should you choose?

For most projects I start with one multilingual dense index plus a keyword index analyzed per language, combined with hybrid search as in A production RAG pipeline, step by step. It keeps the original text, handles cross-lingual questions on the dense side, and lets each language get the right keyword analysis.

Which multilingual embedding models are worth testing?

The MMTEB paper covers more than 500 tasks in over 250 languages, and found that while billion-parameter LLM-based models lead on some subsets, the best publicly available model overall was multilingual-e5-large-instruct with 560 million parameters. Its 512-token limit matters, though: chunks longer than that are silently truncated. BGE-M3 handles up to 8,192 tokens and returns a sparse vector alongside the dense one, which gives you a learned keyword signal without a separate analyzer. The Qwen3 Embedding series comes in three sizes with matching rerankers.

Averages hide the languages you care about. A model that leads a multilingual leaderboard can still be mediocre on Hungarian technical text. Hosted APIs from the big providers are also multilingual; if data must stay in-house, see self-hosting LLMs for GDPR, and for storage, pgvector or a vector database.

How do German compound words break keyword search?

A user searching for Schlauch expects to find Druckluftschlauch. A standard tokenizer sees one token and no match. The fix is decompounding at index time: split compounds into known parts and index both the parts and the whole word. Elasticsearch's hyphenation decompounder uses hyphenation patterns to propose subwords and keeps only those in a word list; its documentation example splits Kaffeetasse into Kaffee and tasse. The filter is built on Lucene's compound word filter, which was designed for Germanic languages.

Dense embeddings handle compounds more gracefully, because the subword tokenizer already splits rare words. That is one more reason to run both: decompounded BM25 for exact parts, embeddings for meaning.

What do Hungarian suffixes and accents need?

For Hungarian, the keyword side needs a stemmer that strips case endings and possessives, so számlán, számlák and számláimat meet at one stem. Snowball has stemmers for both Hungarian and German, and PostgreSQL ships Snowball dictionaries, so plain Postgres full-text search can do it. Add the unaccent extension, because users type szamla for számla and Groesse or Grosse for Größe.

-- PostgreSQL: one tsvector per chunk, analyzed in the chunk's own language
CREATE EXTENSION IF NOT EXISTS unaccent;

-- unaccent() is not IMMUTABLE, so wrap it for use in a generated column
CREATE FUNCTION unaccent_immutable(t text) RETURNS text
  LANGUAGE sql IMMUTABLE PARALLEL SAFE
  AS $$ SELECT public.unaccent('public.unaccent', t) $$;

ALTER TABLE chunks ADD COLUMN lang regconfig NOT NULL DEFAULT 'simple';
ALTER TABLE chunks ADD COLUMN tsv tsvector
  GENERATED ALWAYS AS (to_tsvector(lang, unaccent_immutable(text))) STORED;
CREATE INDEX chunks_tsv ON chunks USING gin (tsv);

-- Query in the user's language: 'hungarian', 'german' or 'english'
SELECT id, ts_rank(tsv, q) AS score
FROM chunks, websearch_to_tsquery('hungarian', unaccent('számláim')) AS q
WHERE tsv @@ q ORDER BY score DESC LIMIT 20;

How should you use language metadata at query time?

Detect and store the language of every chunk at index time, not just of every document; manuals and tickets often mix languages. At query time you have three choices, and it pays to make them explicitly:

• Filter to the user's language when every document exists in every language, as on a fully translated shop. This removes near-duplicates.
• Boost the user's language but allow others, when some content only exists in one language, such as English-only manuals.
• Deduplicate translations of the same source by a shared document id after retrieval, keeping the version in the user's language.

In which language should the model answer?

Tell the model explicitly to answer in the language of the question, because a context full of English chunks pulls answers toward English. Keep quotes and citations in the source language and link the original passage, so a reader can check the claim, as in grounding and citations. Product names, part numbers and legal terms should not be translated at all.

How do you evaluate multilingual retrieval?

Build the eval set from the start with the language mix you expect: questions in German against German documents, in Hungarian against Hungarian ones, and a cross-lingual slice where the answer only exists in another language. Include unaccented and misspelled queries, because that is how people type. Report recall per slice, not one number; the averaged score is exactly what hides a failing language. The metrics themselves are the same as in Evaluating RAG.

# Illustrative eval slices; report recall@10 per slice, never only the average
EVAL = [
  {"q": "Wie lang ist die Garantie auf Druckluftschläuche?", "lang": "de", "gold": ["warranty-de#3"]},
  {"q": "mennyi a garancia a tomlokre",                  "lang": "hu", "gold": ["garancia-hu#2"]},   # no accents
  {"q": "Betriebstemperatur Kompressor K40",              "lang": "de", "gold": ["k40-manual-en#7"]}, # answer only in English
]

Sources

1. Enevoldsen et al.: MMTEB: Massive Multilingual Text Embedding Benchmark (2025)
2. Chen et al.: M3-Embedding (BGE-M3): Multi-Linguality, Multi-Functionality, Multi-Granularity (2024)
3. Zhang et al.: Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models (2025)
4. Hugging Face: intfloat/multilingual-e5-large-instruct model card
5. Elasticsearch Reference: Hyphenation decompounder token filter
6. PostgreSQL 18 Documentation: Text search dictionaries
7. Snowball: stemming algorithms by language