[{"data":1,"prerenderedAt":546},["ShallowReactive",2],{"blog-embedding-quantization-binary-matryoshka-en":3},{"slug":4,"published":5,"minutes":6,"category":7,"tags":8,"keywords":14,"about":25,"sources":35,"cover":45,"og":46,"expertise":47,"locales":48,"lang":49,"title":52,"description":53,"coverAlt":54,"metaTitle":55,"takeaways":56,"faq":62,"toc":78,"blocks":99,"others":309},"embedding-quantization-binary-matryoshka","2026-10-16",5,"rag",[9,10,11,12,13],"Embeddings","Vector search","Quantization","pgvector","RAG",[15,16,17,18,19,20,21,22,23,24],"embedding quantization","binary quantization embeddings","int8 embeddings","matryoshka embeddings","reduce vector database memory","pgvector halfvec","pgvector binary_quantize","embedding dimensions reduction","rescoring binary embeddings","vector index memory cost",[26,29,32],{"name":27,"url":28},"Quantization (signal processing)","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FQuantization_(signal_processing)",{"name":30,"url":31},"Nearest neighbor search","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FNearest_neighbor_search",{"name":33,"url":34},"Hamming distance","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FHamming_distance",[36,39,42],{"title":37,"url":38},"Hugging Face: Binary and Scalar Embedding Quantization for Significantly Faster and Cheaper Retrieval (Mar 2024)","https:\u002F\u002Fhuggingface.co\u002Fblog\u002Fembedding-quantization",{"title":40,"url":41},"Kusupati et al.: Matryoshka Representation Learning (2022)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2205.13147",{"title":43,"url":44},"pgvector README: halfvec, bit, binary quantization and subvector indexing","https:\u002F\u002Fgithub.com\u002Fpgvector\u002Fpgvector","\u002Fimages\u002Fblog\u002Fembedding-quantization-binary-matryoshka\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fembedding-quantization-binary-matryoshka\u002Fog.jpg","ai-engineer",[49,50,51],"en","de","hu","Embedding quantization: binary, int8 and Matryoshka vectors without losing retrieval quality","How binary and int8 quantization and Matryoshka truncation shrink vector indexes 4 to 32 times, what they cost in recall, and how rescoring wins it back.","A float32 embedding shrinking to half precision, int8 and binary bits, with a rescoring step that restores ranking quality.","Embedding quantization: binary and int8 · Balázs Csorba",[57,58,59,60,61],"Binary quantization stores one bit per dimension, 32 times less than float32; int8 stores one byte, 4 times less.","In Hugging Face's tests binary embeddings kept about 92.5% of retrieval performance, and rescoring with the float query raised that to about 96%.","Matryoshka models let you keep only the first dimensions; OpenAI's text-embedding-3-large retained 93.1% at 12 times compression.","pgvector supports halfvec up to 4,000 dimensions, bit up to 64,000 and binary quantization through an expression index with re-ranking.","Quantize the index, keep the full vectors on disk, and measure recall on your own queries before and after.",[63,66,69,72,75],{"q":64,"a":65},"What is embedding quantization?","It stores each embedding value with fewer bits after the model has produced it: 16-bit floats, 8-bit integers or a single bit per dimension. The model is unchanged; only the stored vectors and the index get smaller and faster to compare.",{"q":67,"a":68},"How much retrieval quality does binary quantization lose?","In Hugging Face's experiments with mxbai-embed-large-v1, binary embeddings retained about 92.5% of retrieval performance on MTEB retrieval. Re-scoring the top candidates with the float32 query embedding raised that to about 96%. Other models and domains can differ, so test on your data.",{"q":70,"a":71},"What are Matryoshka embeddings?","Embeddings trained so that the first dimensions carry the most information, which lets you truncate them to a shorter length and keep most of the quality. Models such as OpenAI's text-embedding-3 family and nomic-embed-text-v1.5 support this.",{"q":73,"a":74},"Can I use binary quantization in PostgreSQL?","Yes. pgvector has a binary_quantize function, a bit type and Hamming-distance operators, so you can build an HNSW expression index on the binary version and re-rank the candidates with the original vectors in the same query.",{"q":76,"a":77},"Should I quantize or truncate first?","Start with half precision, which is almost free, then try int8 or binary with rescoring if memory is still the bottleneck. Use Matryoshka truncation when your model supports it and dimension count, not precision, is the limit, for example pgvector's 2,000-dimension index limit for vector.",[79,82,85,88,90,93,96],{"id":80,"title":81},"what-quantization-does","What does embedding quantization actually do?",{"id":83,"title":84},"how-much-quality","How much retrieval quality do you lose?",{"id":86,"title":87},"matryoshka","What do Matryoshka embeddings add?",{"id":12,"title":89},"How do you do this in pgvector?",{"id":91,"title":92},"which-to-choose","Which technique should you choose?",{"id":94,"title":95},"test-plan","How do you test quantization on your own data?",{"id":97,"title":98},"sources","Sources",[100,104,113,116,124,172,175,176,179,188,189,196,199,200,224,226,238,239,265,272,273,289,297,298],{"type":101,"content":102},"paragraph",[103],"Vector indexes get expensive faster than people expect. Ten million chunks with 1,024-dimensional float32 embeddings are about 41 GB of raw vectors before any index overhead, and HNSW indexes want to live in memory. Long before the embedding model or the database becomes the problem, RAM does. Quantization shrinks those vectors by a factor of 2 to 32, and with one extra step costs surprisingly little retrieval quality.",{"type":101,"content":105},[106,107,112],"This article compares half precision, int8, binary quantization and Matryoshka truncation, with the measured retention numbers, shows how to do it in pgvector, and ends with a test plan for your own corpus. The storage choice itself is covered in ",{"tag":108,"to":109,"children":110},"link","\u002Fblog\u002Fpgvector-vs-vector-databases",[111],"pgvector or a vector database",".",{"type":114,"level":115,"id":80,"text":81},"heading",2,{"type":101,"content":117},[118,119,123],"It is a post-processing step on the vectors, not on the model. As the ",{"tag":120,"href":38,"children":121},"a",[122],"Hugging Face write-up"," puts it, binary quantization converts each float32 value to one bit by thresholding at zero, giving a 32 times reduction in memory and storage; scalar int8 quantization maps each value to one of 256 buckets, a 4 times reduction. Binary vectors are compared with Hamming distance, which is very cheap to compute.",{"type":125,"head":126,"rows":135},"table",[127,129,131,133],[128],"Format",[130],"Bytes for 1,024 dims",[132],"10M vectors",[134],"Reduction",[136,145,154,163],[137,139,141,143],[138],"float32",[140],"4,096",[142],"~41 GB",[144],"1×",[146,148,150,152],[147],"float16 (halfvec)",[149],"2,048",[151],"~20 GB",[153],"2×",[155,157,159,161],[156],"int8",[158],"1,024",[160],"~10 GB",[162],"4×",[164,166,168,170],[165],"binary (bit)",[167],"128",[169],"~1.3 GB",[171],"32×",{"type":101,"content":173},[174],"The numbers are raw vector sizes; indexes add overhead on top, roughly in proportion. The point is the shape: binary turns a dedicated-server problem into something that fits next to the application.",{"type":114,"level":115,"id":83,"text":84},{"type":101,"content":177},[178],"Hugging Face measured this with mxbai-embed-large-v1 on MTEB retrieval. Binary embeddings alone kept about 92.5% of the float32 performance. Adding a rescoring step, retrieving candidates with the binary query and then scoring them with the float32 query embedding, raised that to about 96%, while keeping the 32 times memory saving for the index. Int8 sits between the two and is the safer choice when you cannot afford a rescoring pass.",{"type":180,"attrs":181,"inner":185,"caption":186},"diagram",{"viewBox":182,"role":183,"aria-labelledby":184},"0 0 720 170","img","eq1-t eq1-d","\u003Ctitle id=\"eq1-t\">Binary retrieval with rescoring\u003C\u002Ftitle>\u003Cdesc id=\"eq1-d\">The query is embedded once in float32. A binary copy searches the binary index for several times more candidates than needed. The candidates are rescored with the float32 query against their stored full vectors, and the top results go to the reranker or the model.\u003C\u002Fdesc>\u003Cdefs>\u003Cmarker id=\"eq1-ah\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto-start-reverse\">\u003Cpath d=\"M0 0L10 5L0 10z\" class=\"d-head\" \u002F>\u003C\u002Fmarker>\u003C\u002Fdefs>\u003Crect x=\"20\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-box\" \u002F>\u003Ctext x=\"79\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Query\u003C\u002Ftext>\u003Ctext x=\"79\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">float32\u003C\u002Ftext>\u003Cpath d=\"M138 85H158\" class=\"d-line\" marker-end=\"url(#eq1-ah)\" \u002F>\u003Crect x=\"160\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-accent\" \u002F>\u003Ctext x=\"220\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Binary search\u003C\u002Ftext>\u003Ctext x=\"220\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">Hamming, k×4\u003C\u002Ftext>\u003Cpath d=\"M279 85H299\" class=\"d-line\" marker-end=\"url(#eq1-ah)\" \u002F>\u003Crect x=\"301\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-gold\" \u002F>\u003Ctext x=\"360\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Rescore\u003C\u002Ftext>\u003Ctext x=\"360\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">full vectors\u003C\u002Ftext>\u003Cpath d=\"M419 85H439\" class=\"d-line\" marker-end=\"url(#eq1-ah)\" \u002F>\u003Crect x=\"441\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-mint\" \u002F>\u003Ctext x=\"500\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Rerank\u003C\u002Ftext>\u003Ctext x=\"500\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">optional\u003C\u002Ftext>\u003Cpath d=\"M560 85H580\" class=\"d-line\" marker-end=\"url(#eq1-ah)\" \u002F>\u003Crect x=\"582\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-sky\" \u002F>\u003Ctext x=\"641\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Top k\u003C\u002Ftext>\u003Ctext x=\"641\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">to the model\u003C\u002Ftext>",[187],"Only the binary index has to be in memory; full vectors are read for a few hundred candidates per query.",{"type":114,"level":115,"id":86,"text":87},{"type":101,"content":190},[191,192,195],"Quantization reduces the bits per dimension. ",{"tag":120,"href":41,"children":193},[194],"Matryoshka Representation Learning"," reduces the number of dimensions: models are trained so that the first dimensions carry the most information, and a prefix of the vector is a usable embedding on its own. The original paper reports up to 14 times smaller embeddings at the same accuracy on ImageNet classification. For text, the Hugging Face post cites OpenAI's text-embedding-3-large keeping 93.1% of performance at 12 times compression, and nomic-embed-text-v1.5 keeping 95.8% at 3 times and 90% at 6 times.",{"type":101,"content":197},[198],"Truncation only works for models trained this way. Cutting dimensions off an ordinary model degrades it badly, and classic dimensionality reduction such as PCA tends to perform poorly on embeddings, as the same post notes. Check the model card before you slice.",{"type":114,"level":115,"id":12,"text":89},{"type":101,"content":201},[202,203,206,207,211,212,215,216,219,220,223],"The ",{"tag":120,"href":44,"children":204},[205],"pgvector README"," lists the limits for HNSW indexes: ",{"tag":208,"children":209},"code",[210],"vector"," up to 2,000 dimensions, ",{"tag":208,"children":213},[214],"halfvec"," up to 4,000 and ",{"tag":208,"children":217},[218],"bit"," up to 64,000. It supports binary quantization with an expression index on ",{"tag":208,"children":221},[222],"binary_quantize(embedding)"," and Hamming distance, and the same query can re-rank candidates by the original vectors. Subvector expression indexes give you Matryoshka truncation without storing a second column.",{"type":208,"code":225},"-- Binary index for candidate search, full vectors for rescoring\nCREATE INDEX chunks_bq ON chunks\n  USING hnsw ((binary_quantize(embedding)::bit(1024)) bit_hamming_ops);\n\nSELECT id, text FROM (\n  SELECT id, text, embedding FROM chunks\n  ORDER BY binary_quantize(embedding)::bit(1024) \u003C~> binary_quantize($1::vector)\n  LIMIT 200                                   -- 4–10x the final k\n) candidates\nORDER BY embedding \u003C=> $1::vector             -- rescore with the float query\nLIMIT 20;\n\n-- Matryoshka: index only the first 512 dimensions of a model trained for it\nCREATE INDEX chunks_mrl ON chunks\n  USING hnsw ((subvector(embedding, 1, 512)::vector(512)) vector_cosine_ops);\n\n-- Half precision: half the memory, nearly no loss\nCREATE INDEX chunks_half ON chunks\n  USING hnsw ((embedding::halfvec(1024)) halfvec_cosine_ops);\n",{"type":101,"content":227},[228,229,232,233,237],"The query must use the same expression as the index, or Postgres falls back to a sequential scan. Check with ",{"tag":208,"children":230},[231],"EXPLAIN"," once per query shape. Dedicated vector databases such as ",{"tag":108,"to":234,"children":235},"\u002Ftools\u002Fqdrant",[236],"Qdrant"," offer scalar and binary quantization as collection settings with built-in rescoring; the trade-offs are the same.",{"type":114,"level":115,"id":91,"text":92},{"type":125,"head":240,"rows":245},[241,243],[242],"Situation",[244],"Start with",[246,251,255,260],[247,249],[248],"Index fits in memory, want a safe win",[250],"Half precision",[252,254],[253],"Memory is the bottleneck, no rescoring possible",[156],[256,258],[257],"Very large corpus, rescoring is fine",[259],"Binary + float rescoring",[261,263],[262],"Model has more dimensions than the index allows",[264],"Matryoshka truncation, if supported; else halfvec",{"type":266,"variant":267,"title":268,"body":269},"callout","tip","Keep the full vectors",[270],[271],"Quantize the index, not your only copy. Full-precision vectors on disk are what rescoring reads, and they let you switch technique later without re-embedding the corpus, which is the expensive part.",{"type":114,"level":115,"id":94,"text":95},{"type":274,"ordered":275,"items":276},"list",true,[277,283,285,287],[278,279,112],"Take your retrieval eval set, questions with known relevant chunks, as described in ",{"tag":108,"to":280,"children":281},"\u002Fblog\u002Frag-evaluation-metrics",[282],"Evaluating RAG",[284],"Measure recall@10 and recall@50 with float32 as the baseline.",[286],"Repeat for halfvec, int8, binary without rescoring, and binary with rescoring at 4 and 10 times the final k.",[288],"Record index size, query latency at p95 and recall in one table, and pick the cheapest option within your recall budget.",{"type":101,"content":290},[291,292,296],"Run the full pipeline afterwards, including hybrid search and the reranker from ",{"tag":108,"to":293,"children":294},"\u002Fblog\u002Frag-pipeline-chunking-hybrid-search-reranking",[295],"A production RAG pipeline",". A reranker often hides small first-stage recall losses, which is another reason quantized first-stage retrieval works well in practice.",{"type":114,"level":115,"id":97,"text":98},{"type":274,"ordered":275,"items":299},[300,303,306],[301],{"tag":120,"href":38,"children":302},[37],[304],{"tag":120,"href":41,"children":305},[40],[307],{"tag":120,"href":44,"children":308},[43],[310,367,423,477],{"slug":311,"published":5,"minutes":312,"category":7,"tags":313,"keywords":318,"about":329,"sources":339,"cover":361,"og":362,"expertise":47,"locales":363,"lang":49,"title":364,"description":365,"coverAlt":366},"pdf-document-parsing-for-rag",9,[13,314,315,316,317],"PDF parsing","OCR","Docling","Document AI",[319,320,321,322,323,324,325,326,327,328],"pdf parsing for rag","best pdf parser for rag","docling vs marker","marker vs mineru","pdf to markdown for llm","extract tables from pdf for rag","ocr for rag pipeline","olmocr bench","scanned pdf rag","document parsing llm",[330,333,336],{"name":331,"url":332},"PDF","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FPDF",{"name":334,"url":335},"Optical character recognition","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FOptical_character_recognition",{"name":337,"url":338},"Retrieval-augmented generation","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FRetrieval-augmented_generation",[340,343,346,349,352,355,358],{"title":341,"url":342},"Docling: README, features and license (GitHub)","https:\u002F\u002Fgithub.com\u002Fdocling-project\u002Fdocling",{"title":344,"url":345},"Auer et al.: Docling: An Efficient Open-Source Toolkit for AI-driven Document Conversion (2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2501.17887",{"title":347,"url":348},"Datalab: Marker README, olmOCR-bench results and license (GitHub)","https:\u002F\u002Fgithub.com\u002Fdatalab-to\u002Fmarker",{"title":350,"url":351},"OpenDataLab: MinerU README and license (GitHub)","https:\u002F\u002Fgithub.com\u002Fopendatalab\u002FMinerU",{"title":353,"url":354},"Poznanski et al.: olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models (2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2502.18443",{"title":356,"url":357},"Claude Platform Docs: PDF support","https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fbuild-with-claude\u002Fpdf-support",{"title":359,"url":360},"Anthropic: Introducing Contextual Retrieval (Sep 2024)","https:\u002F\u002Fwww.anthropic.com\u002Fnews\u002Fcontextual-retrieval","\u002Fimages\u002Fblog\u002Fpdf-document-parsing-for-rag\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fpdf-document-parsing-for-rag\u002Fog.jpg",[49,50,51],"PDF parsing for RAG: layout pipelines, OCR and vision models compared","Why PDF parsing decides RAG quality, how Docling, Marker, MinerU and VLM OCR differ, and how to test a parser on your own documents.","A PDF page flowing through layout detection, OCR and table recovery into clean Markdown chunks for a retrieval index.",{"slug":368,"published":5,"minutes":369,"category":7,"tags":370,"keywords":374,"about":385,"sources":395,"cover":417,"og":418,"expertise":47,"locales":419,"lang":49,"title":420,"description":421,"coverAlt":422},"multilingual-rag-german-hungarian",7,[13,371,9,372,373],"Multilingual","German","Hungarian",[375,376,377,378,379,380,381,382,383,384],"multilingual rag","german rag embeddings","hungarian nlp search","cross-lingual retrieval","multilingual embedding model","bge-m3 german","german compound words search","hungarian stemming postgres","multilingual-e5-large-instruct","qwen3 embedding multilingual",[386,389,392],{"name":387,"url":388},"Cross-language information retrieval","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FCross-language_information_retrieval",{"name":390,"url":391},"Agglutinative language","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FAgglutinative_language",{"name":393,"url":394},"Compound (linguistics)","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FCompound_(linguistics)",[396,399,402,405,408,411,414],{"title":397,"url":398},"Enevoldsen et al.: MMTEB: Massive Multilingual Text Embedding Benchmark (2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2502.13595",{"title":400,"url":401},"Chen et al.: M3-Embedding (BGE-M3): Multi-Linguality, Multi-Functionality, Multi-Granularity (2024)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2402.03216",{"title":403,"url":404},"Zhang et al.: Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models (2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2506.05176",{"title":406,"url":407},"Hugging Face: intfloat\u002Fmultilingual-e5-large-instruct model card","https:\u002F\u002Fhuggingface.co\u002Fintfloat\u002Fmultilingual-e5-large-instruct",{"title":409,"url":410},"Elasticsearch Reference: Hyphenation decompounder token filter","https:\u002F\u002Fwww.elastic.co\u002Fdocs\u002Freference\u002Ftext-analysis\u002Fanalysis-hyp-decomp-tokenfilter",{"title":412,"url":413},"PostgreSQL 18 Documentation: Text search dictionaries","https:\u002F\u002Fwww.postgresql.org\u002Fdocs\u002Fcurrent\u002Ftextsearch-dictionaries.html",{"title":415,"url":416},"Snowball: stemming algorithms by language","https:\u002F\u002Fsnowballstem.org\u002Falgorithms\u002F","\u002Fimages\u002Fblog\u002Fmultilingual-rag-german-hungarian\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fmultilingual-rag-german-hungarian\u002Fog.jpg",[49,50,51],"Multilingual RAG for German and Hungarian: embeddings, compounds and cross-lingual search","How to build RAG over German and Hungarian content: multilingual embeddings, compound words, stemming, accents and per-language evals.","A German and a Hungarian query meeting in one multilingual vector space next to per-language keyword indexes, then a reranker and an answer.",{"slug":424,"published":5,"minutes":425,"category":7,"tags":426,"keywords":432,"about":443,"sources":452,"cover":471,"og":472,"expertise":47,"locales":473,"lang":49,"title":474,"description":475,"coverAlt":476},"text-to-sql-llm-guardrails",8,[427,428,429,430,431],"Text-to-SQL","LLM security","PostgreSQL","Agents","Analytics",[433,434,435,436,437,438,439,440,441,442],"text to sql llm","text-to-sql guardrails","llm sql agent security","natural language to sql production","read only database role llm","bird benchmark text to sql","spider 2.0 benchmark","sqlglot validate sql","row level security llm","chat with your database",[444,447,450],{"name":445,"url":446},"SQL","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FSQL",{"name":448,"url":449},"Natural language interface to databases","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FNatural-language_user_interface",{"name":429,"url":451},"https:\u002F\u002Fwww.postgresql.org\u002F",[453,456,459,462,465,468],{"title":454,"url":455},"BIRD-bench: leaderboard and dataset description","https:\u002F\u002Fbird-bench.github.io\u002F",{"title":457,"url":458},"Lei et al.: Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows (2024)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2411.07763",{"title":460,"url":461},"PostgreSQL 18 Documentation: Row Security Policies","https:\u002F\u002Fwww.postgresql.org\u002Fdocs\u002Fcurrent\u002Fddl-rowsecurity.html",{"title":463,"url":464},"PostgreSQL 18 Documentation: Client Connection Defaults (read-only and timeouts)","https:\u002F\u002Fwww.postgresql.org\u002Fdocs\u002Fcurrent\u002Fruntime-config-client.html",{"title":466,"url":467},"SQLGlot: SQL parser, transpiler and optimizer (GitHub)","https:\u002F\u002Fgithub.com\u002Ftobymao\u002Fsqlglot",{"title":469,"url":470},"OWASP Gen AI Security Project: LLM06:2025 Excessive Agency","https:\u002F\u002Fgenai.owasp.org\u002Fllmrisk\u002Fllm062025-excessive-agency\u002F","\u002Fimages\u002Fblog\u002Ftext-to-sql-llm-guardrails\u002Fcover.webp","\u002Fimages\u002Fblog\u002Ftext-to-sql-llm-guardrails\u002Fog.jpg",[49,50,51],"Text-to-SQL with LLMs: schema context, read-only roles and validation that holds","How accurate text-to-SQL really is, how to give the model schema context, and the database and code guardrails that keep generated SQL safe.","A question passing through schema context, SQL generation, a validator and a read-only database role before a result table comes back.",{"slug":478,"published":479,"minutes":480,"category":7,"tags":481,"keywords":485,"about":495,"sources":503,"cover":540,"og":541,"expertise":47,"locales":542,"lang":49,"title":543,"description":544,"coverAlt":545},"graphrag-knowledge-graph-rag","2026-10-02",13,[482,483,484,13],"GraphRAG","Knowledge graphs","LightRAG",[482,486,487,488,489,490,491,492,493,494],"knowledge graph RAG","GraphRAG vs vector RAG","Microsoft GraphRAG explained","LightRAG vs GraphRAG","GraphRAG global vs local search","GraphRAG indexing cost","when to use GraphRAG","multi-hop RAG","LazyGraphRAG",[496,499,500],{"name":497,"url":498},"Knowledge graph","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FKnowledge_graph",{"name":337,"url":338},{"name":501,"url":502},"Leiden algorithm","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FLeiden_algorithm",[504,507,510,513,516,519,522,525,528,531,534,537],{"title":505,"url":506},"Edge et al.: From Local to Global: A Graph RAG Approach to Query-Focused Summarization (arXiv:2404.16130)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2404.16130",{"title":508,"url":509},"Microsoft GraphRAG documentation: overview","https:\u002F\u002Fmicrosoft.github.io\u002Fgraphrag\u002F",{"title":511,"url":512},"Microsoft GraphRAG documentation: default dataflow","https:\u002F\u002Fmicrosoft.github.io\u002Fgraphrag\u002Findex\u002Fdefault_dataflow\u002F",{"title":514,"url":515},"Microsoft GraphRAG documentation: global search","https:\u002F\u002Fmicrosoft.github.io\u002Fgraphrag\u002Fquery\u002Fglobal_search\u002F",{"title":517,"url":518},"Microsoft GraphRAG documentation: local search","https:\u002F\u002Fmicrosoft.github.io\u002Fgraphrag\u002Fquery\u002Flocal_search\u002F",{"title":520,"url":521},"Microsoft GraphRAG documentation: DRIFT search","https:\u002F\u002Fmicrosoft.github.io\u002Fgraphrag\u002Fquery\u002Fdrift_search\u002F",{"title":523,"url":524},"Microsoft GraphRAG documentation: getting started","https:\u002F\u002Fmicrosoft.github.io\u002Fgraphrag\u002Fget_started\u002F",{"title":526,"url":527},"Microsoft Research: LazyGraphRAG, setting a new standard for quality and cost (25 November 2024)","https:\u002F\u002Fwww.microsoft.com\u002Fen-us\u002Fresearch\u002Fblog\u002Flazygraphrag-setting-a-new-standard-for-quality-and-cost\u002F",{"title":529,"url":530},"Guo et al.: LightRAG: Simple and Fast Retrieval-Augmented Generation (arXiv:2410.05779)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.05779",{"title":532,"url":533},"HKUDS\u002FLightRAG on GitHub","https:\u002F\u002Fgithub.com\u002FHKUDS\u002FLightRAG",{"title":535,"url":536},"Gutiérrez et al.: HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models (arXiv:2405.14831)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2405.14831",{"title":538,"url":539},"Xiang et al.: When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation (arXiv:2506.05690)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2506.05690","\u002Fimages\u002Fblog\u002Fgraphrag-knowledge-graph-rag\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fgraphrag-knowledge-graph-rag\u002Fog.jpg",[49,50,51],"GraphRAG and knowledge-graph RAG: when a graph beats vector search","What Microsoft GraphRAG and LightRAG really do, what indexing costs, and when a knowledge graph beats vector RAG: multi-hop, global questions, product catalogues.","Diagram: a knowledge graph hub linked to entities, communities, local search, global search and product parts.",1792177152218]