[{"data":1,"prerenderedAt":661},["ShallowReactive",2],{"blog-rag-access-control-permissions-en":3},{"slug":4,"published":5,"minutes":6,"category":7,"tags":8,"keywords":14,"about":27,"sources":37,"cover":62,"og":63,"expertise":64,"locales":65,"lang":66,"title":69,"description":70,"coverAlt":71,"metaTitle":72,"takeaways":73,"faq":79,"toc":95,"blocks":126,"others":423},"rag-access-control-permissions","2026-10-17",9,"rag",[9,10,11,12,13],"RAG","Access control","Security","Multi-tenancy","Vector search",[15,16,17,18,19,20,21,22,23,24,25,26],"rag access control","permission aware rag","document level security rag","rag multi tenancy","vector database access control","pgvector row level security","security trimming azure ai search","openfga rag authorization","qdrant multitenancy","pinecone namespaces multitenancy","owasp llm08 vector and embedding weaknesses","rag data leakage",[28,31,34],{"name":29,"url":30},"Access-control list","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FAccess-control_list",{"name":32,"url":33},"Multitenancy","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FMultitenancy",{"name":35,"url":36},"Retrieval-augmented generation","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FRetrieval-augmented_generation",[38,41,44,47,50,53,56,59],{"title":39,"url":40},"OWASP GenAI Security Project: LLM08:2025 Vector and Embedding Weaknesses","https:\u002F\u002Fgenai.owasp.org\u002Fllmrisk\u002Fllm082025-vector-and-embedding-weaknesses\u002F",{"title":42,"url":43},"PostgreSQL documentation: Row Security Policies","https:\u002F\u002Fwww.postgresql.org\u002Fdocs\u002Fcurrent\u002Fddl-rowsecurity.html",{"title":45,"url":46},"pgvector README: filtering and iterative index scans","https:\u002F\u002Fgithub.com\u002Fpgvector\u002Fpgvector",{"title":48,"url":49},"Qdrant: Multitenancy and partitions guide","https:\u002F\u002Fqdrant.tech\u002Fdocumentation\u002Fguides\u002Fmultiple-partitions\u002F",{"title":51,"url":52},"Pinecone: Implement multitenancy using namespaces","https:\u002F\u002Fdocs.pinecone.io\u002Fguides\u002Findex-data\u002Fimplement-multitenancy",{"title":54,"url":55},"Microsoft Learn: Document-level access control in Azure AI Search","https:\u002F\u002Flearn.microsoft.com\u002Fen-us\u002Fazure\u002Fsearch\u002Fsearch-document-level-access-overview",{"title":57,"url":58},"OpenFGA: RAG authorization use case","https:\u002F\u002Fopenfga.dev\u002Fdocs\u002Fuse-cases\u002Frag-authorization",{"title":60,"url":61},"Morris et al.: Text Embeddings Reveal (Almost) As Much As Text (EMNLP 2023)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2310.06816","\u002Fimages\u002Fblog\u002Frag-access-control-permissions\u002Fcover.webp","\u002Fimages\u002Fblog\u002Frag-access-control-permissions\u002Fog.jpg","ai-engineer",[66,67,68],"en","de","hu","Access control for RAG: permission-aware retrieval without leaking documents","How RAG assistants leak documents and how to stop it: ACLs on every chunk, pre-filtering, row-level security in pgvector, permission sync and leak tests.","Seven cards in a row, six with a lock badge, one bare card sending an arrow outward.","RAG access control: permission-aware search · Balázs Csorba",[74,75,76,77,78],"Enforce permissions in the retrieval query, before any text reaches the model; a system prompt is not an access control.","Every chunk inherits the ACL of its source document, ideally as group IDs, plus a tenant ID and the time the ACL was last synced.","Filtered vector search has a recall trap: pgvector applies the filter after the HNSW scan, so a 10% filter can leave about 4 results unless iterative scans are on.","Permission changes only take effect after they reach the index, so sync ACLs separately from content and check high-risk sources at query time.","Caches, summaries, memory and embeddings are derived data and need the same permissions as the documents they came from.",[80,83,86,89,92],{"q":81,"a":82},"What is permission-aware RAG?","A retrieval-augmented generation system that only retrieves chunks the asking user is allowed to read in the source system. The user's identity is resolved to groups, and those groups filter the search before any text is passed to the model.",{"q":84,"a":85},"Should I filter before or after the vector search?","Before, whenever the store supports it. Pre-filtering in the query keeps unauthorized chunks out of the candidate set entirely. Post-filtering with an authorization service also works, but you have to over-fetch and you risk empty result pages.",{"q":87,"a":88},"Is one index per tenant safer than metadata filters?","It gives stronger isolation: a query cannot reach another tenant's data even if a filter is missing, and offboarding is deleting the namespace. Pinecone recommends one namespace per tenant. Within a tenant you still need document-level filters.",{"q":90,"a":91},"Can embeddings leak the original text?","Yes, to a degree. Morris and colleagues showed in 2023 that an iterative inversion method recovered 92% of 32-token inputs exactly from their embeddings. Treat the vector store with the same protection as the source documents.",{"q":93,"a":94},"How do I test that permissions hold?","Plant canary documents visible to only one group, run a fixed set of queries as users from every group, and assert that no canary chunk appears in any retrieved set it should not. Run it in CI and after every permission sync change.",[96,99,102,105,108,111,114,117,120,123],{"id":97,"title":98},"how-rag-leaks","How does a RAG system leak documents?",{"id":100,"title":101},"where-to-enforce","Where should authorization happen?",{"id":103,"title":104},"patterns","Separate indexes, pre-filters or post-filters?",{"id":106,"title":107},"chunk-metadata","What should each chunk carry?",{"id":109,"title":110},"pgvector-rls","How do you do it in pgvector?",{"id":112,"title":113},"vector-databases","What do Qdrant, Pinecone and Azure AI Search offer?",{"id":115,"title":116},"permission-sync","How do you keep permissions in sync?",{"id":118,"title":119},"derived-data","What about caches, summaries and embeddings?",{"id":121,"title":122},"leak-tests","How do you test that nothing leaks?",{"id":124,"title":125},"sources","Sources",[127,131,134,137,140,169,178,179,182,191,203,204,207,255,263,264,271,273,280,281,288,290,310,311,340,341,344,369,370,378,381,382,385,387,395,396],{"type":128,"content":129},"paragraph",[130],"The fastest way to build an internal RAG assistant is to index everything with one service account and let the model answer. It is also the fastest way to let an apprentice ask for \"this year's salary bands\" and get a cited answer from an HR spreadsheet. The source system had permissions; the index flattened them.",{"type":128,"content":132},[133],"OWASP lists this as LLM08:2025, vector and embedding weaknesses: weak access controls on embeddings, leaks between tenants in shared vector databases, and embedding inversion. This post is the practical side: where to enforce permissions, what each chunk must carry, how to do it in pgvector, Qdrant, Pinecone and Azure AI Search, how to keep permissions in sync, and how to test that nothing leaks.",{"type":135,"level":136,"id":97,"text":98},"heading",2,{"type":128,"content":138},[139],"In the systems I review, leaks rarely come from a clever attack. They come from places where permissions were never attached:",{"type":141,"items":142},"list",[143,149,154,159,164],[144,148],{"tag":145,"children":146},"strong",[147],"An index without ACLs."," Content was crawled with an admin or service account, and chunks carry no information about who may read them.",[150,153],{"tag":145,"children":151},[152],"Filters that fail open."," A missing group claim produces an empty filter, and an empty filter matches everything.",[155,158],{"tag":145,"children":156},[157],"Stale permissions."," Someone was removed from a SharePoint site last week; the index still lists them.",[160,163],{"tag":145,"children":161},[162],"Derived data."," A semantic cache, a conversation memory or a summary that mixes several documents answers a second user from material the first user was allowed to see.",[165,168],{"tag":145,"children":166},[167],"Over-privileged agents."," A retrieval tool runs with the service account rather than the user's identity, so the agent can fetch what the user cannot.",{"type":128,"content":170},[171,172,177],"The last point is the general rule from ",{"tag":173,"to":174,"children":175},"link","\u002Fblog\u002Fai-agent-identity-least-privilege",[176],"AI agents are identities",": retrieval should act on behalf of the user, with the user's rights, not with the rights of the system that hosts it.",{"type":135,"level":136,"id":100,"text":101},{"type":128,"content":180},[181],"In the retrieval query, before anything reaches the model. Once a chunk is in the prompt, no instruction reliably keeps it out of the answer, and prompt injection can ask for it directly. The flow is the same in every stack: authenticate the user, resolve their groups, add those groups as a filter to the search, and pass only the surviving chunks to reranking and generation.",{"type":183,"attrs":184,"inner":188,"caption":189},"diagram",{"viewBox":185,"role":186,"aria-labelledby":187},"0 0 720 170","img","acl1-t acl1-d","\u003Ctitle id=\"acl1-t\">Permission-aware retrieval\u003C\u002Ftitle>\u003Cdesc id=\"acl1-d\">The user token is resolved to a tenant and a list of groups. The search query includes a filter on those groups, so unauthorized chunks are never candidates. Reranking and the language model only see permitted chunks.\u003C\u002Fdesc>\u003Cdefs>\u003Cmarker id=\"acl1-ah\" viewBox=\"0 0 10 10\" refX=\"9\" refY=\"5\" markerWidth=\"7\" markerHeight=\"7\" orient=\"auto-start-reverse\">\u003Cpath d=\"M0 0L10 5L0 10z\" class=\"d-head\" \u002F>\u003C\u002Fmarker>\u003C\u002Fdefs>\u003Crect x=\"20\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-box\" \u002F>\u003Ctext x=\"79\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">User token\u003C\u002Ftext>\u003Ctext x=\"79\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">OIDC, Entra\u003C\u002Ftext>\u003Cpath d=\"M138 85H158\" class=\"d-line\" marker-end=\"url(#acl1-ah)\" \u002F>\u003Crect x=\"160\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-accent\" \u002F>\u003Ctext x=\"220\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Resolve\u003C\u002Ftext>\u003Ctext x=\"220\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">tenant, groups\u003C\u002Ftext>\u003Cpath d=\"M279 85H299\" class=\"d-line\" marker-end=\"url(#acl1-ah)\" \u002F>\u003Crect x=\"301\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-gold\" \u002F>\u003Ctext x=\"360\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Filtered search\u003C\u002Ftext>\u003Ctext x=\"360\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">ACL in query\u003C\u002Ftext>\u003Cpath d=\"M419 85H439\" class=\"d-line\" marker-end=\"url(#acl1-ah)\" \u002F>\u003Crect x=\"441\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-box\" \u002F>\u003Ctext x=\"500\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">Rerank\u003C\u002Ftext>\u003Ctext x=\"500\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">allowed only\u003C\u002Ftext>\u003Cpath d=\"M560 85H580\" class=\"d-line\" marker-end=\"url(#acl1-ah)\" \u002F>\u003Crect x=\"582\" y=\"47\" width=\"118\" height=\"76\" rx=\"10\" class=\"d-mint\" \u002F>\u003Ctext x=\"641\" y=\"81\" text-anchor=\"middle\" class=\"d-text\">LLM\u003C\u002Ftext>\u003Ctext x=\"641\" y=\"103\" text-anchor=\"middle\" class=\"d-small\">cites sources\u003C\u002Ftext>",[190],"If group resolution fails, the request fails; it never falls back to an unfiltered search.",{"type":192,"variant":193,"title":194,"body":195},"callout","warn","The system prompt is not an access control",[196],[197,198,202],"\"Do not reveal HR documents to non-HR staff\" in a system prompt is a hope, not a control. Anything in the context window can end up in the output, especially under the injection patterns described in ",{"tag":173,"to":199,"children":200},"\u002Fblog\u002Fprompt-injection-lethal-trifecta-patterns",[201],"the lethal trifecta",". Keep unauthorized text out of the context instead.",{"type":135,"level":136,"id":103,"text":104},{"type":128,"content":205},[206],"There are four patterns, and most real systems combine two: hard isolation between tenants, and document-level filters inside each tenant.",{"type":208,"head":209,"rows":218},"table",[210,212,214,216],[211],"Pattern",[213],"How it works",[215],"Strength",[217],"Weakness",[219,228,237,246],[220,222,224,226],[221],"Index or namespace per tenant",[223],"Queries target one tenant's partition",[225],"A missing filter cannot cross tenants; delete to offboard",[227],"No document-level rights inside a tenant",[229,231,233,235],[230],"Metadata pre-filter",[232],"Allowed groups stored per chunk, filter in the query",[234],"Unauthorized chunks never become candidates",[236],"ACLs must be synced into the index",[238,240,242,244],[239],"Post-filter with authorization service",[241],"Retrieve candidates, then check each against OpenFGA or similar",[243],"Always current permissions",[245],"Over-fetching, empty pages, latency",[247,249,251,253],[248],"Database row-level security",[250],"Postgres policies filter rows for the session's role",[252],"Default deny, enforced below the application",[254],"Only for Postgres; recall trap with ANN indexes",{"type":128,"content":256},[257,258,262],"OpenFGA describes the post-filter variants: check each candidate for a ",{"tag":259,"children":260},"code",[261],"can_view"," relation, or call list-objects once to get every document the user can read and intersect it with the candidates. It recommends per-document checks for small sets and list-objects when the set is larger. The cost of any post-filter is that you have to retrieve more than you need, because some candidates will be dropped.",{"type":135,"level":136,"id":106,"text":107},{"type":128,"content":265},[266,267,270],"Chunks inherit the permissions of their source document. Store groups rather than individual users where possible; Microsoft gives the same advice for ACL-secured content in Azure AI Search, and Pinecone calls filtering by large lists of user IDs an anti-pattern, with each ",{"tag":259,"children":268},[269],"$in"," operator limited to 10,000 values. A minimal payload:",{"type":259,"code":272},"{\n  \"chunk_id\": \"sp-hr-0412#7\",\n  \"tenant_id\": \"acme\",\n  \"doc_id\": \"sp-hr-0412\",\n  \"source\": \"sharepoint:\u002F\u002Fhr\u002Fsalary-bands-2026.xlsx\",\n  \"allowed_groups\": [\"grp-hr\", \"grp-exec\"],\n  \"classification\": \"confidential\",\n  \"acl_version\": \"2026-10-16T22:05:11Z\",\n  \"text\": \"...\"\n}",{"type":128,"content":274},[275,276,279],"The ",{"tag":259,"children":277},[278],"acl_version"," field is what makes sync auditable: you can see how old each chunk's permissions are and find chunks that missed an update. For anything derived from several documents, such as a summary, use the intersection of their groups, not the union.",{"type":135,"level":136,"id":109,"text":110},{"type":128,"content":282},[283,284,287],"Postgres row-level security puts the filter below the application. Once enabled on a table, all normal access must be allowed by a policy, and with no policy the default is deny. Superusers, roles with BYPASSRLS and normally the table owner bypass it, so the application must connect as a separate role, and ",{"tag":259,"children":285},[286],"FORCE ROW LEVEL SECURITY"," makes the owner subject to the policy too.",{"type":259,"code":289},"ALTER TABLE chunks ENABLE ROW LEVEL SECURITY;\nALTER TABLE chunks FORCE ROW LEVEL SECURITY;\n\n-- The app sets the caller's tenant and groups per transaction.\nCREATE POLICY chunk_read ON chunks FOR SELECT TO rag_app\n  USING (\n    tenant_id = current_setting('app.tenant_id')\n    AND allowed_groups && string_to_array(current_setting('app.groups'), ',')\n  );\n\nBEGIN;\nSET LOCAL app.tenant_id = 'acme';\nSET LOCAL app.groups = 'grp-sales,grp-all';\nSET LOCAL hnsw.iterative_scan = relaxed_order;   -- pgvector 0.8.0+\nSELECT chunk_id, text FROM chunks\nORDER BY embedding \u003C=> $1 LIMIT 20;\nCOMMIT;",{"type":128,"content":291},[292,293,296,297,300,301,304,305,309],"The last setting matters more than it looks. pgvector applies filters after the approximate index is scanned, and with the default ",{"tag":259,"children":294},[295],"hnsw.ef_search"," of 40, a filter that matches 10% of rows leaves about 4 results. A permission filter is exactly such a filter, and the symptom is not a leak but an assistant that \"cannot find\" documents for users with few rights. Iterative index scans, added in pgvector 0.8.0, keep scanning until enough rows match, up to ",{"tag":259,"children":298},[299],"hnsw.max_scan_tuples"," (20,000 by default). For tenants, a B-tree index on ",{"tag":259,"children":302},[303],"tenant_id"," or list partitioning by tenant also helps. My comparison of ",{"tag":173,"to":306,"children":307},"\u002Fblog\u002Fpgvector-vs-vector-databases",[308],"pgvector and vector databases"," covers when this is enough.",{"type":135,"level":136,"id":112,"text":113},{"type":141,"items":312},[313,326,331],[314,317,318,321,322,325],{"tag":145,"children":315},[316],"Qdrant"," recommends one collection with a tenant field over a collection per tenant. A keyword payload index with ",{"tag":259,"children":319},[320],"is_tenant"," set to true (from v1.11) stores each tenant's vectors together, every query carries a ",{"tag":259,"children":323},[324],"must"," filter on the tenant, and from v1.16 tiered multitenancy promotes large tenants to their own shard. Document-level groups go into the payload as another filter.",[327,330],{"tag":145,"children":328},[329],"Pinecone"," recommends one namespace per tenant in a serverless index. Namespaces are stored separately, which Pinecone describes as physical isolation, and offboarding is deleting the namespace. Queries are billed by namespace size, so filtering a shared 100 GB namespace costs 100 times more than querying a 1 GB tenant namespace.",[332,335,336,339],{"tag":145,"children":333},[334],"Azure AI Search"," supports security filters, plain string comparison on an identity field that your application fills, and in preview native document-level ACLs and RBAC scopes, SharePoint ACLs and Purview sensitivity labels, enforced by passing the user's Entra token in the ",{"tag":259,"children":337},[338],"x-ms-query-source-authorization"," header. Microsoft is explicit that security filters are string matching, not authentication, so your application must verify the identity first.",{"type":135,"level":136,"id":115,"text":116},{"type":128,"content":342},[343],"An index is a copy, and copies go stale. Microsoft's documentation says it directly: permission changes in the source system only appear in search results after the metadata is synchronized to the index, and for SharePoint, changes inherited from a parent site, library or folder need an explicit refresh. Every other stack has the same property, documented or not. Three measures keep the window small:",{"type":141,"ordered":345,"items":346},true,[347,359,364],[348,351,352,355,356,358],{"tag":145,"children":349},[350],"Sync ACLs separately from content."," Updating ",{"tag":259,"children":353},[354],"allowed_groups"," is a metadata write, not a re-embed. Run it far more often than content indexing, and alert when ",{"tag":259,"children":357},[278]," falls behind.",[360,363],{"tag":145,"children":361},[362],"Resolve groups at query time."," Store groups on chunks and resolve the user's current group membership from the identity provider per request, so removing someone from a group takes effect immediately.",[365,368],{"tag":145,"children":366},[367],"Re-check high-risk sources live."," For HR, legal or board material, add a post-filter against the source system or an authorization service for the final candidates, accepting the extra latency for those collections only.",{"type":135,"level":136,"id":118,"text":119},{"type":128,"content":371},[372,373,377],"Everything computed from documents inherits their sensitivity. A semantic cache must include the user's permission set in its key, or one user's answer is served to another; I described the scoping in ",{"tag":173,"to":374,"children":375},"\u002Fblog\u002Fsemantic-caching-llm-responses",[376],"semantic caching",". Conversation memory belongs to one user. Summaries, entity descriptions and chunk contexts generated across several documents get the intersection of their groups.",{"type":128,"content":379},[380],"Embeddings are not anonymous either. Morris and colleagues showed in 2023 that an iterative inversion method recovered 92% of 32-token inputs exactly from their embeddings, and recovered full names from clinical notes. Give the vector store the same network isolation, encryption and access logging as the source documents, and include it in deletion requests.",{"type":135,"level":136,"id":121,"text":122},{"type":128,"content":383},[384],"Test the retrieval layer, not the answers. A model may decline to quote a chunk it received, which hides the bug. Plant canary documents with unique marker strings, each visible to exactly one group, and assert on the retrieved chunk IDs:",{"type":259,"code":386},"import pytest\n\nPERSONAS = {\"sales\": [\"grp-sales\"], \"hr\": [\"grp-hr\"], \"none\": []}\nCANARIES = {\"canary-hr\": \"grp-hr\", \"canary-sales\": \"grp-sales\"}\nQUERIES = [\"salary bands\", \"canary report\", \"confidential\", \"*\"]\n\n@pytest.mark.parametrize(\"persona,groups\", PERSONAS.items())\n@pytest.mark.parametrize(\"query\", QUERIES)\ndef test_no_canary_leaks(persona, groups, query):\n    hits = retrieve(query, tenant=\"acme\", groups=groups, k=50)\n    for doc_id, owner_group in CANARIES.items():\n        if owner_group not in groups:\n            assert all(h.doc_id != doc_id for h in hits), (persona, query, doc_id)\n\ndef test_missing_groups_fail_closed():\n    with pytest.raises(PermissionError):\n        retrieve(\"anything\", tenant=\"acme\", groups=None, k=10)",{"type":128,"content":388},[389,390,394],"Add two more checks: revocation latency, by removing a test user from a group and measuring how long until the canary disappears, and tenant isolation, by querying with a valid token for tenant A against tenant B's namespace. Run the suite in CI and after every change to the sync job. Access control belongs next to the defences against ",{"tag":173,"to":391,"children":392},"\u002Fblog\u002Frag-poisoning-knowledge-base-attacks",[393],"RAG poisoning",": both protect the knowledge base, one against reading, the other against writing.",{"type":135,"level":136,"id":124,"text":125},{"type":141,"ordered":345,"items":397},[398,402,405,408,411,414,417,420],[399],{"tag":400,"href":40,"children":401},"a",[39],[403],{"tag":400,"href":43,"children":404},[42],[406],{"tag":400,"href":46,"children":407},[45],[409],{"tag":400,"href":49,"children":410},[48],[412],{"tag":400,"href":52,"children":413},[51],[415],{"tag":400,"href":55,"children":416},[54],[418],{"tag":400,"href":58,"children":419},[57],[421],{"tag":400,"href":61,"children":422},[60],[424,496,554,607],{"slug":425,"published":5,"minutes":6,"category":7,"tags":426,"keywords":430,"about":443,"sources":453,"cover":490,"og":491,"expertise":64,"locales":492,"lang":66,"title":493,"description":494,"coverAlt":495},"embedding-model-selection-2026",[427,9,428,13,429],"Embeddings","MTEB","Model selection",[431,432,433,434,435,436,437,438,439,440,441,442],"best embedding model 2026","how to choose an embedding model","embedding model comparison","MTEB leaderboard explained","RTEB benchmark","open source embedding model","multilingual embedding model","voyage-4 vs openai embeddings","Qwen3-Embedding","EmbeddingGemma","embedding model for RAG","switching embedding models re-embed",[444,447,450],{"name":445,"url":446},"Word embedding","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FWord_embedding",{"name":448,"url":449},"Information retrieval","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FInformation_retrieval",{"name":451,"url":452},"Discounted cumulative gain","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FDiscounted_cumulative_gain",[454,457,460,463,466,469,472,475,478,481,484,487],{"title":455,"url":456},"Enevoldsen et al.: MMTEB, Massive Multilingual Text Embedding Benchmark (2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2502.13595",{"title":458,"url":459},"Hugging Face: Introducing RTEB, a new standard for retrieval evaluation (Oct 2025)","https:\u002F\u002Fhuggingface.co\u002Fblog\u002Frteb",{"title":461,"url":462},"OpenAI: Vector embeddings guide","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fguides\u002Fembeddings",{"title":464,"url":465},"OpenAI: API pricing","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fpricing",{"title":467,"url":468},"Voyage AI: The Voyage 4 model family, shared embedding space with MoE architecture (Jan 2026)","https:\u002F\u002Fblog.voyageai.com\u002F2026\u002F01\u002F15\u002Fvoyage-4\u002F",{"title":470,"url":471},"Voyage AI: Text embeddings documentation","https:\u002F\u002Fdocs.voyageai.com\u002Fdocs\u002Fembeddings",{"title":473,"url":474},"Voyage AI: Pricing","https:\u002F\u002Fdocs.voyageai.com\u002Fdocs\u002Fpricing",{"title":476,"url":477},"Google: Gemini API embeddings documentation","https:\u002F\u002Fai.google.dev\u002Fgemini-api\u002Fdocs\u002Fembeddings",{"title":479,"url":480},"Google: Gemini API pricing","https:\u002F\u002Fai.google.dev\u002Fgemini-api\u002Fdocs\u002Fpricing",{"title":482,"url":483},"Cohere: Embed models documentation","https:\u002F\u002Fdocs.cohere.com\u002Fdocs\u002Fcohere-embed",{"title":485,"url":486},"Qwen: Qwen3-Embedding-8B model card","https:\u002F\u002Fhuggingface.co\u002FQwen\u002FQwen3-Embedding-8B",{"title":488,"url":489},"Google Developers Blog: Introducing EmbeddingGemma (Sep 2025)","https:\u002F\u002Fdevelopers.googleblog.com\u002Fen\u002Fintroducing-embeddinggemma\u002F","\u002Fimages\u002Fblog\u002Fembedding-model-selection-2026\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fembedding-model-selection-2026\u002Fog.jpg",[66,67,68],"Choosing an embedding model in 2026: leaderboards, costs and a bake-off on your own data","How to pick an embedding model for RAG in 2026: what MTEB and RTEB tell you, hosted vs open weights, prices, dimensions, and a one-week bake-off.","Three columns of stacked blocks, the middle tallest, each measured by a short bar and a wide bracket.",{"slug":497,"published":5,"minutes":498,"category":7,"tags":499,"keywords":503,"about":515,"sources":523,"cover":548,"og":549,"expertise":64,"locales":550,"lang":66,"title":551,"description":552,"coverAlt":553},"contextual-retrieval-late-chunking",10,[9,500,427,501,502],"Chunking","Contextual retrieval","Late chunking",[504,505,506,507,508,509,510,511,512,513,514],"contextual retrieval","anthropic contextual retrieval","late chunking","contextual chunk embeddings","contextual embeddings rag","contextual bm25","chunk context rag","voyage-context-3","voyage-context-4","late chunking vs contextual retrieval","rag chunking context loss",[516,517,520],{"name":35,"url":36},{"name":518,"url":519},"Okapi BM25","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FOkapi_BM25",{"name":521,"url":522},"Sentence embedding","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FSentence_embedding",[524,527,530,533,536,539,542,545],{"title":525,"url":526},"Anthropic: Introducing Contextual Retrieval (Sep 2024)","https:\u002F\u002Fwww.anthropic.com\u002Fnews\u002Fcontextual-retrieval",{"title":528,"url":529},"Günther et al.: Late Chunking, Contextual Chunk Embeddings Using Long-Context Embedding Models (2024, rev. 2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2409.04701",{"title":531,"url":532},"Jina AI: Late Chunking in Long-Context Embedding Models (Aug 2024)","https:\u002F\u002Fjina.ai\u002Fnews\u002Flate-chunking-in-long-context-embedding-models\u002F",{"title":534,"url":535},"Voyage AI: Introducing voyage-context-3 (Jul 2025)","https:\u002F\u002Fblog.voyageai.com\u002F2025\u002F07\u002F23\u002Fvoyage-context-3\u002F",{"title":537,"url":538},"Voyage AI: voyage-context-4 (Jun 2026)","https:\u002F\u002Fblog.voyageai.com\u002F2026\u002F06\u002F29\u002Fvoyage-context-4\u002F",{"title":540,"url":541},"Voyage AI: Contextualized chunk embeddings documentation","https:\u002F\u002Fdocs.voyageai.com\u002Fdocs\u002Fcontextualized-chunk-embeddings",{"title":543,"url":544},"Anthropic: Claude API pricing","https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fabout-claude\u002Fpricing",{"title":546,"url":547},"Qu, Tu and Bao: Is Semantic Chunking Worth the Computational Cost? (2024)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2410.13070","\u002Fimages\u002Fblog\u002Fcontextual-retrieval-late-chunking\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fcontextual-retrieval-late-chunking\u002Fog.jpg",[66,67,68],"Contextual retrieval and late chunking: giving RAG chunks back their context","Chunks lose their document. How contextual retrieval, late chunking and contextualized embedding models fix that, what each costs, and when to skip it.","Four loose square pieces gathered by a wide bracket below, each joined upward to a large sheet.",{"slug":555,"published":556,"minutes":6,"category":7,"tags":557,"keywords":562,"about":573,"sources":581,"cover":601,"og":602,"expertise":64,"locales":603,"lang":66,"title":604,"description":605,"coverAlt":606},"pdf-document-parsing-for-rag","2026-10-16",[9,558,559,560,561],"PDF parsing","OCR","Docling","Document AI",[563,564,565,566,567,568,569,570,571,572],"pdf parsing for rag","best pdf parser for rag","docling vs marker","marker vs mineru","pdf to markdown for llm","extract tables from pdf for rag","ocr for rag pipeline","olmocr bench","scanned pdf rag","document parsing llm",[574,577,580],{"name":575,"url":576},"PDF","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FPDF",{"name":578,"url":579},"Optical character recognition","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FOptical_character_recognition",{"name":35,"url":36},[582,585,588,591,594,597,600],{"title":583,"url":584},"Docling: README, features and license (GitHub)","https:\u002F\u002Fgithub.com\u002Fdocling-project\u002Fdocling",{"title":586,"url":587},"Auer et al.: Docling: An Efficient Open-Source Toolkit for AI-driven Document Conversion (2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2501.17887",{"title":589,"url":590},"Datalab: Marker README, olmOCR-bench results and license (GitHub)","https:\u002F\u002Fgithub.com\u002Fdatalab-to\u002Fmarker",{"title":592,"url":593},"OpenDataLab: MinerU README and license (GitHub)","https:\u002F\u002Fgithub.com\u002Fopendatalab\u002FMinerU",{"title":595,"url":596},"Poznanski et al.: olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models (2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2502.18443",{"title":598,"url":599},"Claude Platform Docs: PDF support","https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fbuild-with-claude\u002Fpdf-support",{"title":525,"url":526},"\u002Fimages\u002Fblog\u002Fpdf-document-parsing-for-rag\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fpdf-document-parsing-for-rag\u002Fog.jpg",[66,67,68],"PDF parsing for RAG: layout pipelines, OCR and vision models compared","Why PDF parsing decides RAG quality, how Docling, Marker, MinerU and VLM OCR differ, and how to test a parser on your own documents.","Scattered uneven blocks on a sheet becoming an even row of rectangles on a second sheet.",{"slug":608,"published":556,"minutes":609,"category":7,"tags":610,"keywords":614,"about":624,"sources":634,"cover":655,"og":656,"expertise":64,"locales":657,"lang":66,"title":658,"description":659,"coverAlt":660},"multilingual-rag-german-hungarian",7,[9,611,427,612,613],"Multilingual","German","Hungarian",[615,616,617,618,437,619,620,621,622,623],"multilingual rag","german rag embeddings","hungarian nlp search","cross-lingual retrieval","bge-m3 german","german compound words search","hungarian stemming postgres","multilingual-e5-large-instruct","qwen3 embedding multilingual",[625,628,631],{"name":626,"url":627},"Cross-language information retrieval","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FCross-language_information_retrieval",{"name":629,"url":630},"Agglutinative language","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FAgglutinative_language",{"name":632,"url":633},"Compound (linguistics)","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FCompound_(linguistics)",[635,637,640,643,646,649,652],{"title":636,"url":456},"Enevoldsen et al.: MMTEB: Massive Multilingual Text Embedding Benchmark (2025)",{"title":638,"url":639},"Chen et al.: M3-Embedding (BGE-M3): Multi-Linguality, Multi-Functionality, Multi-Granularity (2024)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2402.03216",{"title":641,"url":642},"Zhang et al.: Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models (2025)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2506.05176",{"title":644,"url":645},"Hugging Face: intfloat\u002Fmultilingual-e5-large-instruct model card","https:\u002F\u002Fhuggingface.co\u002Fintfloat\u002Fmultilingual-e5-large-instruct",{"title":647,"url":648},"Elasticsearch Reference: Hyphenation decompounder token filter","https:\u002F\u002Fwww.elastic.co\u002Fdocs\u002Freference\u002Ftext-analysis\u002Fanalysis-hyp-decomp-tokenfilter",{"title":650,"url":651},"PostgreSQL 18 Documentation: Text search dictionaries","https:\u002F\u002Fwww.postgresql.org\u002Fdocs\u002Fcurrent\u002Ftextsearch-dictionaries.html",{"title":653,"url":654},"Snowball: stemming algorithms by language","https:\u002F\u002Fsnowballstem.org\u002Falgorithms\u002F","\u002Fimages\u002Fblog\u002Fmultilingual-rag-german-hungarian\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fmultilingual-rag-german-hungarian\u002Fog.jpg",[66,67,68],"Multilingual RAG for German and Hungarian: embeddings, compounds and cross-lingual search","How to build RAG over German and Hungarian content: multilingual embeddings, compound words, stemming, accents and per-language evals.","Two rows of small squares, each with a dot above, merging into one tall rectangle.",1792222776086]