[{"data":1,"prerenderedAt":667},["ShallowReactive",2],{"blog-multilingual-rag-german-hungarian-en":3},{"slug":4,"published":5,"minutes":6,"category":7,"tags":8,"keywords":14,"about":25,"sources":35,"cover":57,"og":58,"expertise":59,"locales":60,"lang":61,"title":64,"description":65,"coverAlt":66,"metaTitle":67,"takeaways":68,"faq":74,"toc":90,"blocks":118,"others":442},"multilingual-rag-german-hungarian","2026-10-16",7,"rag",[9,10,11,12,13],"RAG","Multilingual","Embeddings","German","Hungarian",[15,16,17,18,19,20,21,22,23,24],"multilingual rag","german rag embeddings","hungarian nlp search","cross-lingual retrieval","multilingual embedding model","bge-m3 german","german compound words search","hungarian stemming postgres","multilingual-e5-large-instruct","qwen3 embedding multilingual",[26,29,32],{"name":27,"url":28},"Cross-language information retrieval","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FCross-language_information_retrieval",{"name":30,"url":31},"Agglutinative language","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FAgglutinative_language",{"name":33,"url":34},"Compound (linguistics)","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FCompound_(linguistics)",[36,39,42,45,48,51,54],{"title":37,"url":38},"Enevoldsen et al.: MMTEB: Massive Multilingual Text Embedding Benchmark (2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2502.13595",{"title":40,"url":41},"Chen et al.: M3-Embedding (BGE-M3): Multi-Linguality, Multi-Functionality, Multi-Granularity (2024)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2402.03216",{"title":43,"url":44},"Zhang et al.: Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models (2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2506.05176",{"title":46,"url":47},"Hugging Face: intfloat\u002Fmultilingual-e5-large-instruct model card","https:\u002F\u002Fhuggingface.co\u002Fintfloat\u002Fmultilingual-e5-large-instruct",{"title":49,"url":50},"Elasticsearch Reference: Hyphenation decompounder token filter","https:\u002F\u002Fwww.elastic.co\u002Fdocs\u002Freference\u002Ftext-analysis\u002Fanalysis-hyp-decomp-tokenfilter",{"title":52,"url":53},"PostgreSQL 18 Documentation: Text search dictionaries","https:\u002F\u002Fwww.postgresql.org\u002Fdocs\u002Fcurrent\u002Ftextsearch-dictionaries.html",{"title":55,"url":56},"Snowball: stemming algorithms by language","https:\u002F\u002Fsnowballstem.org\u002Falgorithms\u002F","\u002Fimages\u002Fblog\u002Fmultilingual-rag-german-hungarian\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fmultilingual-rag-german-hungarian\u002Fog.jpg","ai-engineer",[61,62,63],"en","de","hu","Multilingual RAG for German and Hungarian: embeddings, compounds and cross-lingual search","How to build RAG over German and Hungarian content: multilingual embeddings, compound words, stemming, accents and per-language evals.","A German and a Hungarian query meeting in one multilingual vector space next to per-language keyword indexes, then a reranker and an answer.","Multilingual RAG for German and Hungarian · Balázs Csorba",[69,70,71,72,73],"A multilingual embedding model lets a German question find a Hungarian or English passage, but keyword search still needs per-language analysis.","German compounds and Hungarian suffixes break naive BM25: use decompounding for German and a Snowball stemmer for Hungarian.","Users type Hungarian and German without accents and umlauts; fold diacritics in the keyword index as well as storing the original text.","Store the language of every chunk and decide explicitly whether to filter, boost or mix languages at query time.","Leaderboards average across many languages; build a small eval set per language and a cross-lingual one before picking a model.",[75,78,81,84,87],{"q":76,"a":77},"Which embedding model is best for German and Hungarian?","Test on your own data. The MMTEB authors found the 560M-parameter multilingual-e5-large-instruct to be the best publicly available model on their multilingual benchmarks, while BGE-M3 and the Qwen3 Embedding series are strong newer options with longer inputs. Per-language scores can differ a lot from the average.",{"q":79,"a":80},"Should I translate everything into English before indexing?","Usually not. Translation adds cost and errors, loses exact terms and makes citations awkward. A multilingual embedding model plus per-language keyword search keeps the original text. Translation can still help for a small, high-value corpus in a language your models handle poorly.",{"q":82,"a":83},"How do I handle German compound words in search?","Split compounds into their parts at index time with a decompounding filter, such as the Elasticsearch hyphenation decompounder with a German word list, and keep the full compound too. Dense embeddings cope with compounds better, which is another reason to use hybrid search.",{"q":85,"a":86},"Does PostgreSQL support Hungarian full-text search?","Yes. PostgreSQL ships Snowball stemmers, and Snowball has stemmers for both Hungarian and German, so you can index with to_tsvector using a hungarian or german configuration and add the unaccent extension for accent-insensitive matching.",{"q":88,"a":89},"In which language should the assistant answer?","In the language of the question, unless the user asks otherwise. Quote and cite the source in its original language, because a translated quote can no longer be checked against the document.",[91,94,97,100,103,106,109,112,115],{"id":92,"title":93},"why-multilingual-rag-is-harder","Why is multilingual RAG harder than English RAG?",{"id":95,"title":96},"index-strategies","Which index strategy should you choose?",{"id":98,"title":99},"multilingual-embedding-models","Which multilingual embedding models are worth testing?",{"id":101,"title":102},"german-compounds","How do German compound words break keyword search?",{"id":104,"title":105},"hungarian-suffixes-and-accents","What do Hungarian suffixes and accents need?",{"id":107,"title":108},"language-metadata","How should you use language metadata at query time?",{"id":110,"title":111},"answer-language","In which language should the model answer?",{"id":113,"title":114},"evaluate-per-language","How do you evaluate multilingual retrieval?",{"id":116,"title":117},"sources","Sources",[119,123,126,129,151,152,200,209,218,219,256,270,283,284,309,312,313,352,354,375,376,379,398,399,407,408,415,417,418],{"type":120,"content":121},"paragraph",[122],"Most RAG tutorials assume English documents and English questions. Projects in Austria, Germany and Hungary rarely look like that: product data in German, support tickets in Hungarian, manuals in English, and users who ask in whichever language they think in, often without umlauts or accents. Retrieval that works in a demo quietly drops half the relevant passages.",{"type":120,"content":124},[125],"This article covers the decisions that matter for German and Hungarian content: index strategy, multilingual embedding models, what German compounds and Hungarian suffixes do to keyword search, accent folding, language metadata, the answer language, and how to evaluate per language. The same patterns carry over to other morphologically rich languages.",{"type":127,"level":128,"id":92,"text":93},"heading",2,{"type":120,"content":130},[131,132,136,137,140,141,136,144,136,147,150],"Three problems stack up. First, morphology: German glues nouns into compounds (",{"tag":133,"children":134},"em",[135],"Kaffeetasse",", ",{"tag":133,"children":138},[139],"Druckluftschlauch","), and Hungarian is agglutinative, so one stem appears with many suffixes (",{"tag":133,"children":142},[143],"ház",{"tag":133,"children":145},[146],"házban",{"tag":133,"children":148},[149],"házainkban",": house, in the house, in our houses). Exact-match search misses most of those forms. Second, cross-lingual queries: a German question whose answer sits in an English manual finds nothing with keyword search. Third, mixed corpora: the same document in three languages can crowd the top results with near-duplicates.",{"type":127,"level":128,"id":95,"text":96},{"type":153,"head":154,"rows":163},"table",[155,157,159,161],[156],"Strategy",[158],"How it works",[160],"Good for",[162],"Watch out",[164,173,182,191],[165,167,169,171],[166],"One multilingual index",[168],"One embedding model for all languages, language stored as metadata",[170],"Cross-lingual questions, mixed corpora",[172],"Keyword side still needs per-language analysis",[174,176,178,180],[175],"Index per language",[177],"Separate indexes, query routed by detected language",[179],"Fully localized sites with parallel content",[181],"Misses answers that only exist in another language",[183,185,187,189],[184],"Translate at index time",[186],"Everything translated into one pivot language",[188],"Small, valuable corpora in weakly supported languages",[190],"Cost, translation errors, lost exact terms",[192,194,196,198],[193],"Translate the query",[195],"Query rewritten into each corpus language before search",[197],"Keyword-heavy search across languages",[199],"An extra LLM call per query",{"type":120,"content":201},[202,203,208],"For most projects I start with one multilingual dense index plus a keyword index analyzed per language, combined with hybrid search as in ",{"tag":204,"to":205,"children":206},"link","\u002Fblog\u002Frag-pipeline-chunking-hybrid-search-reranking",[207],"A production RAG pipeline, step by step",". It keeps the original text, handles cross-lingual questions on the dense side, and lets each language get the right keyword analysis.",{"type":210,"attrs":211,"inner":215,"caption":216},"diagram",{"viewBox":212,"role":213,"aria-labelledby":214},"0 0 720 170","img","mlr1-t mlr1-d","\u003Ctitle id=\"mlr1-t\">Hybrid multilingual retrieval\u003C\u002Ftitle>\u003Cdesc id=\"mlr1-d\">The query language is detected. The query goes to a multilingual dense index and to keyword indexes analyzed per language. Results are fused, reranked by a multilingual reranker, and the answer is written in the language of the question with citations in the original language.\u003C\u002Fdesc>\u003Cdefs>\u003Cmarker id=\"mlr1-ah\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto-start-reverse\">\u003Cpath d=\"M0 0L10 5L0 10z\" class=\"d-head\" \u002F>\u003C\u002Fmarker>\u003C\u002Fdefs>\u003Crect x=\"20\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-box\" \u002F>\u003Ctext x=\"79\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Query\u003C\u002Ftext>\u003Ctext x=\"79\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">detect language\u003C\u002Ftext>\u003Cpath d=\"M138 85H158\" class=\"d-line\" marker-end=\"url(#mlr1-ah)\" \u002F>\u003Crect x=\"160\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-accent\" \u002F>\u003Ctext x=\"220\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Dense\u003C\u002Ftext>\u003Ctext x=\"220\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">multilingual\u003C\u002Ftext>\u003Cpath d=\"M279 85H299\" class=\"d-line\" marker-end=\"url(#mlr1-ah)\" \u002F>\u003Crect x=\"301\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-gold\" \u002F>\u003Ctext x=\"360\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Keyword\u003C\u002Ftext>\u003Ctext x=\"360\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">per language\u003C\u002Ftext>\u003Cpath d=\"M419 85H439\" class=\"d-line\" marker-end=\"url(#mlr1-ah)\" \u002F>\u003Crect x=\"441\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-mint\" \u002F>\u003Ctext x=\"500\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Rerank\u003C\u002Ftext>\u003Ctext x=\"500\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">multilingual\u003C\u002Ftext>\u003Cpath d=\"M560 85H580\" class=\"d-line\" marker-end=\"url(#mlr1-ah)\" \u002F>\u003Crect x=\"582\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-sky\" \u002F>\u003Ctext x=\"641\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Answer\u003C\u002Ftext>\u003Ctext x=\"641\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">user language\u003C\u002Ftext>",[217],"The dense side bridges languages; the keyword side catches exact terms such as part numbers and names.",{"type":127,"level":128,"id":98,"text":99},{"type":153,"head":220,"rows":229},[221,223,225,227],[222],"Model",[224],"Size",[226],"Max input",[228],"Notes",[230,238,247],[231,232,234,236],[23],[233],"560M",[235],"512 tokens",[237],"Best public model in the MMTEB paper; 100 languages from XLM-R",[239,241,243,245],[240],"BGE-M3",[242],"XLM-R based",[244],"8,192 tokens",[246],"100+ languages; dense, sparse and multi-vector output from one model",[248,250,252,254],[249],"Qwen3 Embedding",[251],"0.6B \u002F 4B \u002F 8B",[253],"Long",[255],"Strong on multilingual and cross-lingual retrieval; matching rerankers",{"type":120,"content":257},[258,259,263,264,266,267,269],"The ",{"tag":260,"href":38,"children":261},"a",[262],"MMTEB paper"," covers more than 500 tasks in over 250 languages, and found that while billion-parameter LLM-based models lead on some subsets, the best publicly available model overall was multilingual-e5-large-instruct with 560 million parameters. Its 512-token limit matters, though: chunks longer than that are silently truncated. ",{"tag":260,"href":41,"children":265},[240]," handles up to 8,192 tokens and returns a sparse vector alongside the dense one, which gives you a learned keyword signal without a separate analyzer. The ",{"tag":260,"href":44,"children":268},[249]," series comes in three sizes with matching rerankers.",{"type":120,"content":271},[272,273,277,278,282],"Averages hide the languages you care about. A model that leads a multilingual leaderboard can still be mediocre on Hungarian technical text. Hosted APIs from the big providers are also multilingual; if data must stay in-house, see ",{"tag":204,"to":274,"children":275},"\u002Fblog\u002Fself-hosted-llm-gdpr-cost",[276],"self-hosting LLMs for GDPR",", and for storage, ",{"tag":204,"to":279,"children":280},"\u002Fblog\u002Fpgvector-vs-vector-databases",[281],"pgvector or a vector database",".",{"type":127,"level":128,"id":101,"text":102},{"type":120,"content":285},[286,287,290,291,293,294,297,298,300,301,304,305,308],"A user searching for ",{"tag":133,"children":288},[289],"Schlauch"," expects to find ",{"tag":133,"children":292},[139],". A standard tokenizer sees one token and no match. The fix is decompounding at index time: split compounds into known parts and index both the parts and the whole word. Elasticsearch's ",{"tag":260,"href":50,"children":295},[296],"hyphenation decompounder"," uses hyphenation patterns to propose subwords and keeps only those in a word list; its documentation example splits ",{"tag":133,"children":299},[135]," into ",{"tag":133,"children":302},[303],"Kaffee"," and ",{"tag":133,"children":306},[307],"tasse",". The filter is built on Lucene's compound word filter, which was designed for Germanic languages.",{"type":120,"content":310},[311],"Dense embeddings handle compounds more gracefully, because the subword tokenizer already splits rare words. That is one more reason to run both: decompounded BM25 for exact parts, embeddings for meaning.",{"type":127,"level":128,"id":104,"text":105},{"type":120,"content":314},[315,316,136,319,304,322,325,326,329,330,334,335,338,339,304,342,345,346,338,349,282],"For Hungarian, the keyword side needs a stemmer that strips case endings and possessives, so ",{"tag":133,"children":317},[318],"számlán",{"tag":133,"children":320},[321],"számlák",{"tag":133,"children":323},[324],"számláimat"," meet at one stem. ",{"tag":260,"href":56,"children":327},[328],"Snowball"," has stemmers for both Hungarian and German, and PostgreSQL ships Snowball dictionaries, so plain Postgres full-text search can do it. Add the ",{"tag":331,"children":332},"code",[333],"unaccent"," extension, because users type ",{"tag":133,"children":336},[337],"szamla"," for ",{"tag":133,"children":340},[341],"számla",{"tag":133,"children":343},[344],"Groesse"," or ",{"tag":133,"children":347},[348],"Grosse",{"tag":133,"children":350},[351],"Größe",{"type":331,"code":353},"-- PostgreSQL: one tsvector per chunk, analyzed in the chunk's own language\nCREATE EXTENSION IF NOT EXISTS unaccent;\n\n-- unaccent() is not IMMUTABLE, so wrap it for use in a generated column\nCREATE FUNCTION unaccent_immutable(t text) RETURNS text\n  LANGUAGE sql IMMUTABLE PARALLEL SAFE\n  AS $$ SELECT public.unaccent('public.unaccent', t) $$;\n\nALTER TABLE chunks ADD COLUMN lang regconfig NOT NULL DEFAULT 'simple';\nALTER TABLE chunks ADD COLUMN tsv tsvector\n  GENERATED ALWAYS AS (to_tsvector(lang, unaccent_immutable(text))) STORED;\nCREATE INDEX chunks_tsv ON chunks USING gin (tsv);\n\n-- Query in the user's language: 'hungarian', 'german' or 'english'\nSELECT id, ts_rank(tsv, q) AS score\nFROM chunks, websearch_to_tsquery('hungarian', unaccent('számláim')) AS q\nWHERE tsv @@ q ORDER BY score DESC LIMIT 20;\n",{"type":355,"variant":356,"title":357,"body":358},"callout","warn","Watch for broken Hungarian characters",[359],[360,361,304,364,367,368,304,371,374],"Older Hungarian documents often contain ",{"tag":133,"children":362},[363],"õ",{"tag":133,"children":365},[366],"û"," (tilde) instead of ",{"tag":133,"children":369},[370],"ő",{"tag":133,"children":372},[373],"ű"," (double acute), a leftover of Latin-1 encodings. Normalize them before indexing, or stemming, embeddings and exact matches all treat them as different words.",{"type":127,"level":128,"id":107,"text":108},{"type":120,"content":377},[378],"Detect and store the language of every chunk at index time, not just of every document; manuals and tickets often mix languages. At query time you have three choices, and it pays to make them explicitly:",{"type":380,"items":381},"list",[382,388,393],[383,387],{"tag":384,"children":385},"strong",[386],"Filter"," to the user's language when every document exists in every language, as on a fully translated shop. This removes near-duplicates.",[389,392],{"tag":384,"children":390},[391],"Boost"," the user's language but allow others, when some content only exists in one language, such as English-only manuals.",[394,397],{"tag":384,"children":395},[396],"Deduplicate"," translations of the same source by a shared document id after retrieval, keeping the version in the user's language.",{"type":127,"level":128,"id":110,"text":111},{"type":120,"content":400},[401,402,406],"Tell the model explicitly to answer in the language of the question, because a context full of English chunks pulls answers toward English. Keep quotes and citations in the source language and link the original passage, so a reader can check the claim, as in ",{"tag":204,"to":403,"children":404},"\u002Fblog\u002Fllm-hallucination-grounding-citations",[405],"grounding and citations",". Product names, part numbers and legal terms should not be translated at all.",{"type":127,"level":128,"id":113,"text":114},{"type":120,"content":409},[410,411,282],"Build the eval set from the start with the language mix you expect: questions in German against German documents, in Hungarian against Hungarian ones, and a cross-lingual slice where the answer only exists in another language. Include unaccented and misspelled queries, because that is how people type. Report recall per slice, not one number; the averaged score is exactly what hides a failing language. The metrics themselves are the same as in ",{"tag":204,"to":412,"children":413},"\u002Fblog\u002Frag-evaluation-metrics",[414],"Evaluating RAG",{"type":331,"code":416},"# Illustrative eval slices; report recall@10 per slice, never only the average\nEVAL = [\n  {\"q\": \"Wie lang ist die Garantie auf Druckluftschläuche?\", \"lang\": \"de\", \"gold\": [\"warranty-de#3\"]},\n  {\"q\": \"mennyi a garancia a tomlokre\",                  \"lang\": \"hu\", \"gold\": [\"garancia-hu#2\"]},   # no accents\n  {\"q\": \"Betriebstemperatur Kompressor K40\",              \"lang\": \"de\", \"gold\": [\"k40-manual-en#7\"]}, # answer only in English\n]\n",{"type":127,"level":128,"id":116,"text":117},{"type":380,"ordered":419,"items":420},true,[421,424,427,430,433,436,439],[422],{"tag":260,"href":38,"children":423},[37],[425],{"tag":260,"href":41,"children":426},[40],[428],{"tag":260,"href":44,"children":429},[43],[431],{"tag":260,"href":47,"children":432},[46],[434],{"tag":260,"href":50,"children":435},[49],[437],{"tag":260,"href":53,"children":438},[52],[440],{"tag":260,"href":56,"children":441},[55],[443,500,554,598],{"slug":444,"published":5,"minutes":445,"category":7,"tags":446,"keywords":451,"about":462,"sources":472,"cover":494,"og":495,"expertise":59,"locales":496,"lang":61,"title":497,"description":498,"coverAlt":499},"pdf-document-parsing-for-rag",9,[9,447,448,449,450],"PDF parsing","OCR","Docling","Document AI",[452,453,454,455,456,457,458,459,460,461],"pdf parsing for rag","best pdf parser for rag","docling vs marker","marker vs mineru","pdf to markdown for llm","extract tables from pdf for rag","ocr for rag pipeline","olmocr bench","scanned pdf rag","document parsing llm",[463,466,469],{"name":464,"url":465},"PDF","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FPDF",{"name":467,"url":468},"Optical character recognition","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FOptical_character_recognition",{"name":470,"url":471},"Retrieval-augmented generation","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FRetrieval-augmented_generation",[473,476,479,482,485,488,491],{"title":474,"url":475},"Docling: README, features and license (GitHub)","https:\u002F\u002Fgithub.com\u002Fdocling-project\u002Fdocling",{"title":477,"url":478},"Auer et al.: Docling: An Efficient Open-Source Toolkit for AI-driven Document Conversion (2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2501.17887",{"title":480,"url":481},"Datalab: Marker README, olmOCR-bench results and license (GitHub)","https:\u002F\u002Fgithub.com\u002Fdatalab-to\u002Fmarker",{"title":483,"url":484},"OpenDataLab: MinerU README and license (GitHub)","https:\u002F\u002Fgithub.com\u002Fopendatalab\u002FMinerU",{"title":486,"url":487},"Poznanski et al.: olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models (2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2502.18443",{"title":489,"url":490},"Claude Platform Docs: PDF support","https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fbuild-with-claude\u002Fpdf-support",{"title":492,"url":493},"Anthropic: Introducing Contextual Retrieval (Sep 2024)","https:\u002F\u002Fwww.anthropic.com\u002Fnews\u002Fcontextual-retrieval","\u002Fimages\u002Fblog\u002Fpdf-document-parsing-for-rag\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fpdf-document-parsing-for-rag\u002Fog.jpg",[61,62,63],"PDF parsing for RAG: layout pipelines, OCR and vision models compared","Why PDF parsing decides RAG quality, how Docling, Marker, MinerU and VLM OCR differ, and how to test a parser on your own documents.","A PDF page flowing through layout detection, OCR and table recovery into clean Markdown chunks for a retrieval index.",{"slug":501,"published":5,"minutes":502,"category":7,"tags":503,"keywords":509,"about":520,"sources":529,"cover":548,"og":549,"expertise":59,"locales":550,"lang":61,"title":551,"description":552,"coverAlt":553},"text-to-sql-llm-guardrails",8,[504,505,506,507,508],"Text-to-SQL","LLM security","PostgreSQL","Agents","Analytics",[510,511,512,513,514,515,516,517,518,519],"text to sql llm","text-to-sql guardrails","llm sql agent security","natural language to sql production","read only database role llm","bird benchmark text to sql","spider 2.0 benchmark","sqlglot validate sql","row level security llm","chat with your database",[521,524,527],{"name":522,"url":523},"SQL","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FSQL",{"name":525,"url":526},"Natural language interface to databases","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FNatural-language_user_interface",{"name":506,"url":528},"https:\u002F\u002Fwww.postgresql.org\u002F",[530,533,536,539,542,545],{"title":531,"url":532},"BIRD-bench: leaderboard and dataset description","https:\u002F\u002Fbird-bench.github.io\u002F",{"title":534,"url":535},"Lei et al.: Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows (2024)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2411.07763",{"title":537,"url":538},"PostgreSQL 18 Documentation: Row Security Policies","https:\u002F\u002Fwww.postgresql.org\u002Fdocs\u002Fcurrent\u002Fddl-rowsecurity.html",{"title":540,"url":541},"PostgreSQL 18 Documentation: Client Connection Defaults (read-only and timeouts)","https:\u002F\u002Fwww.postgresql.org\u002Fdocs\u002Fcurrent\u002Fruntime-config-client.html",{"title":543,"url":544},"SQLGlot: SQL parser, transpiler and optimizer (GitHub)","https:\u002F\u002Fgithub.com\u002Ftobymao\u002Fsqlglot",{"title":546,"url":547},"OWASP Gen AI Security Project: LLM06:2025 Excessive Agency","https:\u002F\u002Fgenai.owasp.org\u002Fllmrisk\u002Fllm062025-excessive-agency\u002F","\u002Fimages\u002Fblog\u002Ftext-to-sql-llm-guardrails\u002Fcover.webp","\u002Fimages\u002Fblog\u002Ftext-to-sql-llm-guardrails\u002Fog.jpg",[61,62,63],"Text-to-SQL with LLMs: schema context, read-only roles and validation that holds","How accurate text-to-SQL really is, how to give the model schema context, and the database and code guardrails that keep generated SQL safe.","A question passing through schema context, SQL generation, a validator and a read-only database role before a result table comes back.",{"slug":555,"published":5,"minutes":556,"category":7,"tags":557,"keywords":561,"about":572,"sources":582,"cover":592,"og":593,"expertise":59,"locales":594,"lang":61,"title":595,"description":596,"coverAlt":597},"embedding-quantization-binary-matryoshka",5,[11,558,559,560,9],"Vector search","Quantization","pgvector",[562,563,564,565,566,567,568,569,570,571],"embedding quantization","binary quantization embeddings","int8 embeddings","matryoshka embeddings","reduce vector database memory","pgvector halfvec","pgvector binary_quantize","embedding dimensions reduction","rescoring binary embeddings","vector index memory cost",[573,576,579],{"name":574,"url":575},"Quantization (signal processing)","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FQuantization_(signal_processing)",{"name":577,"url":578},"Nearest neighbor search","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FNearest_neighbor_search",{"name":580,"url":581},"Hamming distance","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FHamming_distance",[583,586,589],{"title":584,"url":585},"Hugging Face: Binary and Scalar Embedding Quantization for Significantly Faster and Cheaper Retrieval (Mar 2024)","https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fembedding-quantization",{"title":587,"url":588},"Kusupati et al.: Matryoshka Representation Learning (2022)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2205.13147",{"title":590,"url":591},"pgvector README: halfvec, bit, binary quantization and subvector indexing","https:\u002F\u002Fgithub.com\u002Fpgvector\u002Fpgvector","\u002Fimages\u002Fblog\u002Fembedding-quantization-binary-matryoshka\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fembedding-quantization-binary-matryoshka\u002Fog.jpg",[61,62,63],"Embedding quantization: binary, int8 and Matryoshka vectors without losing retrieval quality","How binary and int8 quantization and Matryoshka truncation shrink vector indexes 4 to 32 times, what they cost in recall, and how rescoring wins it back.","A float32 embedding shrinking to half precision, int8 and binary bits, with a rescoring step that restores ranking quality.",{"slug":599,"published":600,"minutes":601,"category":7,"tags":602,"keywords":606,"about":616,"sources":624,"cover":661,"og":662,"expertise":59,"locales":663,"lang":61,"title":664,"description":665,"coverAlt":666},"graphrag-knowledge-graph-rag","2026-10-02",13,[603,604,605,9],"GraphRAG","Knowledge graphs","LightRAG",[603,607,608,609,610,611,612,613,614,615],"knowledge graph RAG","GraphRAG vs vector RAG","Microsoft GraphRAG explained","LightRAG vs GraphRAG","GraphRAG global vs local search","GraphRAG indexing cost","when to use GraphRAG","multi-hop RAG","LazyGraphRAG",[617,620,621],{"name":618,"url":619},"Knowledge graph","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FKnowledge_graph",{"name":470,"url":471},{"name":622,"url":623},"Leiden algorithm","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FLeiden_algorithm",[625,628,631,634,637,640,643,646,649,652,655,658],{"title":626,"url":627},"Edge et al.: From Local to Global: A Graph RAG Approach to Query-Focused Summarization (arXiv:2404.16130)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2404.16130",{"title":629,"url":630},"Microsoft GraphRAG documentation: overview","https:\u002F\u002Fmicrosoft.github.io\u002Fgraphrag\u002F",{"title":632,"url":633},"Microsoft GraphRAG documentation: default dataflow","https:\u002F\u002Fmicrosoft.github.io\u002Fgraphrag\u002Findex\u002Fdefault_dataflow\u002F",{"title":635,"url":636},"Microsoft GraphRAG documentation: global search","https:\u002F\u002Fmicrosoft.github.io\u002Fgraphrag\u002Fquery\u002Fglobal_search\u002F",{"title":638,"url":639},"Microsoft GraphRAG documentation: local search","https:\u002F\u002Fmicrosoft.github.io\u002Fgraphrag\u002Fquery\u002Flocal_search\u002F",{"title":641,"url":642},"Microsoft GraphRAG documentation: DRIFT search","https:\u002F\u002Fmicrosoft.github.io\u002Fgraphrag\u002Fquery\u002Fdrift_search\u002F",{"title":644,"url":645},"Microsoft GraphRAG documentation: getting started","https:\u002F\u002Fmicrosoft.github.io\u002Fgraphrag\u002Fget_started\u002F",{"title":647,"url":648},"Microsoft Research: LazyGraphRAG, setting a new standard for quality and cost (25 November 2024)","https:\u002F\u002Fwww.microsoft.com\u002Fen-us\u002Fresearch\u002Fblog\u002Flazygraphrag-setting-a-new-standard-for-quality-and-cost\u002F",{"title":650,"url":651},"Guo et al.: LightRAG: Simple and Fast Retrieval-Augmented Generation (arXiv:2410.05779)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.05779",{"title":653,"url":654},"HKUDS\u002FLightRAG on GitHub","https:\u002F\u002Fgithub.com\u002FHKUDS\u002FLightRAG",{"title":656,"url":657},"Gutiérrez et al.: HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models (arXiv:2405.14831)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2405.14831",{"title":659,"url":660},"Xiang et al.: When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation (arXiv:2506.05690)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2506.05690","\u002Fimages\u002Fblog\u002Fgraphrag-knowledge-graph-rag\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fgraphrag-knowledge-graph-rag\u002Fog.jpg",[61,62,63],"GraphRAG and knowledge-graph RAG: when a graph beats vector search","What Microsoft GraphRAG and LightRAG really do, what indexing costs, and when a knowledge graph beats vector RAG: multi-hop, global questions, product catalogues.","Diagram: a knowledge graph hub linked to entities, communities, local search, global search and product parts.",1792177152157]