[{"data":1,"prerenderedAt":1081},["ShallowReactive",2],{"blog-deepseek-v4-1-flash-1017-en":3},{"slug":4,"published":5,"minutes":6,"category":7,"tags":8,"keywords":14,"about":28,"sources":36,"cover":55,"og":56,"expertise":57,"locales":58,"lang":59,"title":62,"description":63,"coverAlt":64,"metaTitle":65,"takeaways":66,"faq":72,"toc":85,"blocks":125,"others":824},"deepseek-v4-1-flash-1017","2026-10-17",15,"llmops",[9,10,11,12,13],"DeepSeek","Open-weight models","KV cache","Mixture of experts","LLM cost",[15,16,17,18,19,20,21,22,23,24,25,26,27],"DeepSeek V4.1 Flash 1017","DeepSeek V4.1 Flash","Causal Encoder-Decoder","Compressed Sparse Attention 2","CSA2","FP4 KV cache","KV cache compression","SWA Bounded Replay","Engram conditional memory","DSpark speculative decoding","DeepSeek pricing","DeepSWE benchmark","open-weight coding model",[29,31,33],{"name":9,"url":30},"https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FDeepSeek",{"name":12,"url":32},"https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FMixture_of_experts",{"name":34,"url":35},"Transformer (deep learning architecture)","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FTransformer_(deep_learning_architecture)",[37,40,43,46,49,52],{"title":38,"url":39},"DeepSeek-AI: DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression (arXiv:2609.19969)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.19969",{"title":41,"url":42},"Hugging Face: deepseek-ai\u002FDeepSeek-V4.1-Flash-1017 model card","https:\u002F\u002Fhuggingface.co\u002Fdeepseek-ai\u002FDeepSeek-V4.1-Flash-1017",{"title":44,"url":45},"OpenRouter: DeepSeek V4.1 Flash 1017, pricing and providers","https:\u002F\u002Fopenrouter.ai\u002Fdeepseek\u002Fdeepseek-v4.1-flash-1017",{"title":47,"url":48},"The Decoder: Deepseek V4.1 Flash 1017 beats larger rivals in agentic tasks while once again massively cutting per-task costs","https:\u002F\u002Fthe-decoder.com\u002Fdeepseek-v4-1-flash-1017-beats-larger-rivals-in-agentic-tasks-while-once-again-massively-cutting-per-task-costs\u002F",{"title":50,"url":51},"AI Weekly: DeepSeek opens V4.1-Flash-1017 weights and $0.75 Search API","https:\u002F\u002Faiweekly.co\u002Falerts\u002Fdeepseek-opens-v41-flash-1017-weights-and-075-search-api",{"title":53,"url":54},"DeepSeek on X: DeepSeek-V4.1-Flash-1017 announcement","https:\u002F\u002Fx.com\u002Fdeepseek_ai\u002Fstatus\u002F2111375640717033777","\u002Fimages\u002Fblog\u002Fdeepseek-v4-1-flash-1017\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fdeepseek-v4-1-flash-1017\u002Fog.jpg","ai-engineer",[59,60,61],"en","de","hu","DeepSeek V4.1 Flash 1017: 890 bytes of KV cache per token, and what that buys an agent","A technical deep dive into DeepSeek V4.1 Flash 1017: the encoder-decoder split, CSA2 cache sharing, FP4 KV cache, the benchmarks and the real cost per agent task.","A tall stack of thin cards on the left, an arrow, and the same cards pressed into a stack a quarter of the height on the right.","DeepSeek V4.1 Flash 1017 deep dive · Balázs Csorba",[67,68,69,70,71],"DeepSeek V4.1 Flash 1017, released on 17 October 2026, is a post-training update of V4.1 Flash: same 552B-parameter backbone, 8B active parameters during prefill and 16B during decode, 1M-token context, MIT-licensed weights.","The architecture is built around the KV cache: a causal encoder-decoder split, cross-layer cache sharing (CSA2) and an FP4 main cache bring the always-in-HBM cache to 890 bytes per token, about a quarter of V4 Flash.","On DeepSeek's own numbers, 1017 scores 77.0 on DeepSWE v1.1 (Opus 5: 74.0) and 42.3 on Terminal-Bench 4.0 (up from 31.2), but still trails Opus 5 clearly on HLE, Terminal-Bench 4.0 and ProgramBench.","Pricing is $0.15 input, $0.60 output and $0.003 cache reads per million tokens, doubled during two weekday peak windows in UTC. Cache hits are 50 times cheaper than fresh input, so prefix stability decides your bill.","The release is mostly a cost story: DeepSeek's chart puts 1017 at just under $0.20 per DeepSWE task, against about $0.50 for the previous Flash and $2 to over $10 for the closed frontier models.",[73,76,79,82],{"q":74,"a":75},"What is DeepSeek V4.1 Flash 1017?","It is the 17 October 2026 snapshot of DeepSeek V4.1 Flash, an open-weight multimodal mixture-of-experts model with 552B backbone parameters, 196B Engram memory parameters and a 1M-token context. DeepSeek calls it a minor update: same architecture, new post-training, better benchmark scores and shorter outputs.",{"q":77,"a":78},"Why does it activate 8B parameters on input but 16B on output?","Because of the Causal Encoder-Decoder design. The decoder's global KV cache is projected from the last encoder layer, so prompt tokens only need to run through the 20 encoder layers during prefill. Generated tokens run through all 40 layers. For input-heavy agent workloads that nearly halves prefill compute.",{"q":80,"a":81},"How much does DeepSeek V4.1 Flash 1017 cost?","As listed on OpenRouter on 17 October 2026: $0.15 per million input tokens, $0.60 per million output tokens and $0.003 per million cached input tokens. On weekdays between 01:00 and 04:00 UTC and between 06:00 and 10:00 UTC all three prices double.",{"q":83,"a":84},"Can I run DeepSeek V4.1 Flash 1017 myself?","Yes, the weights are on Hugging Face and ModelScope under the MIT license, with inference code in the repository. In practice it is a multi-GPU deployment: around 750B parameters including the Engram tables, which DeepSeek's own stack keeps in host memory and prefetches over RDMA.",[86,89,92,95,98,101,104,107,110,113,116,119,122],{"id":87,"title":88},"at-a-glance","The model at a glance",{"id":90,"title":91},"why-kv-cache","Why the KV cache is the real bottleneck",{"id":93,"title":94},"causal-encoder-decoder","Causal Encoder-Decoder: why prefill only uses 8B parameters",{"id":96,"title":97},"csa2","CSA2: most layers do not keep their own cache",{"id":99,"title":100},"fp4-and-890-bytes","FP4 KV cache and the 890-byte budget",{"id":102,"title":103},"other-components","Engram, DSpark and single-pass mHC",{"id":105,"title":106},"training","Training: 45T tokens, sparse from step one",{"id":108,"title":109},"what-1017-changes","What the 1017 update changes",{"id":111,"title":112},"cost-per-task","The real headline: cost per task",{"id":114,"title":115},"using-it","Calling it and running it",{"id":117,"title":118},"limitations","Limitations worth knowing",{"id":120,"title":121},"what-i-would-do","What I would do with it",{"id":123,"title":124},"sources","Sources",[126,144,151,158,161,219,220,229,232,233,244,247,254,255,261,264,283,286,329,340,343,346,347,354,364,367,370,371,374,391,394,395,398,405,407,415,430,431,442,449,648,651,657,658,665,675,687,694,695,718,720,733,735,736,756,757,783,791,792],{"type":127,"content":128},"paragraph",[129,130,134,135,139,140,143],"DeepSeek shipped ",{"tag":131,"children":132},"strong",[133],"V4.1 Flash 1017"," this morning: a dated snapshot of its V4.1 Flash model, live on the DeepSeek API, on ",{"tag":136,"href":45,"children":137},"a",[138],"OpenRouter"," and as MIT-licensed weights on ",{"tag":136,"href":42,"children":141},[142],"Hugging Face",". The headline in most coverage is a benchmark: 77.0 on DeepSWE v1.1, ahead of Claude Opus 5. That number is real, but it is the least interesting thing about the model.",{"type":127,"content":145},[146,147,150],"The interesting part is the architecture underneath, described in DeepSeek's ",{"tag":136,"href":39,"children":148},[149],"V4.1 Flash technical report",". Almost every design decision in it answers one question: how small can the KV cache get before quality drops? The answer is 890 bytes per token. This article walks through how they got there, what the 1017 update changes on top, what the benchmarks do and do not say, and what it costs to run an agent on it.",{"type":152,"variant":153,"title":154,"body":155},"callout","note","Two documents, one model",[156],[157],"The architecture comes from the V4.1 Flash technical report (arXiv, 17 September 2026). The 1017 model card calls the new release a minor update to V4.1 Flash, so the architecture is unchanged and the differences are in post-training. Everything about 1017 specifically comes from the model card and launch coverage, not from the paper.",{"type":159,"level":160,"id":87,"text":88},"heading",2,{"type":162,"head":163,"rows":168},"table",[164,166],[165],"Property",[167],"Value",[169,174,179,184,189,194,199,204,209,214],[170,172],[171],"Type",[173],"Multimodal mixture-of-experts transformer, text and image in, text out",[175,177],[176],"Backbone parameters",[178],"552B",[180,182],[181],"Engram memory parameters",[183],"196B",[185,187],[186],"Active per token",[188],"8B in prefill, 16B in decode",[190,192],[191],"Layers",[193],"40: 20-layer causal encoder, 20-layer decoder",[195,197],[196],"Experts per MoE layer",[198],"1 shared, 384 routed, 6 active",[200,202],[201],"Context",[203],"1M tokens (1,048,576)",[205,207],[206],"Global KV cache",[208],"890 bytes per token",[210,212],[211],"Pre-training",[213],"45T multimodal tokens",[215,217],[216],"License",[218],"MIT",{"type":159,"level":160,"id":90,"text":91},{"type":127,"content":221},[222,223,228],"A coding agent does not look like a chatbot. Every turn resends a long and growing history, tool results get appended, and the model writes comparatively little. The workload is input-heavy, and the expensive parts are the ones that scale with input: prefill compute for tokens that miss the cache, and the memory to keep the KV cache of all those tokens around. If you have read my piece on ",{"tag":224,"to":225,"children":226},"link","\u002Fblog\u002Fcontext-rot-compaction-long-running-agents",[227],"context rot and compaction",", this is the infrastructure side of the same problem.",{"type":127,"content":230},[231],"DeepSeek V4 already split attention into two branches: a sliding-window branch (SWA) that only sees recent tokens and costs a fixed amount of memory, and a compressed global branch that sees the whole context. For long sequences the global branch dominates memory, and it has to live in GPU memory (HBM). On top of that, caches persisted for prefix reuse live on SSD or in host memory, and moving them costs bandwidth. V4.1 Flash attacks all three: compute, HBM and persistent storage.",{"type":159,"level":160,"id":93,"text":94},{"type":127,"content":234},[235,236,239,240,243],"The first idea is borrowed from YOCO (You Only Cache Once) and extended. The 40 layers are split into a 20-layer ",{"tag":131,"children":237},[238],"causal encoder"," and a 20-layer ",{"tag":131,"children":241},[242],"decoder",". In a normal transformer every layer computes its own keys and values from its own hidden state. Here, the decoder layers do not: their global KV entries are projected directly from the hidden state of the last encoder layer, with a separate learned projection per decoder layer.",{"type":127,"content":245},[246],"The consequence is the asymmetric parameter count. For a prompt token, the model only needs to run the encoder half to produce every global KV entry the decoder will later read, which is why prefill activates 8B parameters per token. A generated token has to go through all 40 layers, so decode activates 16B. The paper puts prefill complexity at O(N·L\u002F2) instead of O(N·L): roughly half the compute for every token that misses the cache. In an agent loop, that is every tool result.",{"type":127,"content":248},[249,250,253],"There is one catch. The sliding-window branch still computes keys and values per layer from that layer's own hidden state, which keeps local processing deep. To fill the decoder's SWA cache exactly you would have to replay window × L\u002F2 tokens through the decoder. DeepSeek's answer is ",{"tag":131,"children":251},[252],"Decoder SWA Bounded Replay",": replay only the last 128 tokens (the window size) and accept an approximate state. The paper reports a negligible quality impact, and the replay is simulated during post-training so the model learns to live with it.",{"type":159,"level":160,"id":96,"text":97},{"type":127,"content":256},[257,258,260],"You can shrink a KV cache along three axes: make each entry smaller (GQA, or MLA with its shared latent), store fewer entries along the sequence (compress every m tokens into one), or store fewer copies across layers. ",{"tag":131,"children":259},[18]," is DeepSeek's attempt to use all three at once. Like V4's CSA, each layer has a lightweight indexer that scores the compressed global KV entries and picks the top 512 for each query; the query then attends to those 512 entries plus its local window.",{"type":127,"content":262},[263],"The new part is cross-layer reuse. Every CSA2 layer is statically assigned one of three modes:",{"type":265,"ordered":266,"items":267},"list",false,[268,273,278],[269,272],{"tag":131,"children":270},[271],"Full mode"," computes its own main KV, projects the indexer keys from it, and runs the indexer to select fresh top-512 indices. This is what a V4 CSA layer did.",[274,277],{"tag":131,"children":275},[276],"Reindex mode"," reuses the main KV and indexer keys of the last Full layer, but computes its own indexer query and selects its own top-512. The cache is shared; the selection can differ.",[279,282],{"tag":131,"children":280},[281],"Reuse mode"," reuses both the cache and the latest indices, and goes straight to sparse attention. No indexer work at all.",{"type":127,"content":284},[285],"In all three modes each layer still computes its own query and its own sliding-window KV, so layers do not become copies of each other. Here is how the 40 layers are actually laid out:",{"type":162,"head":287,"rows":295},[288,290,291,293],[289],"Block",[191],[292],"Attention",[294],"Layout",[296,305,313,322],[297,299,301,303],[298],"Encoder",[300],"0–1",[302],"SWA only",[304],"—",[306,307,309,311],[298],[308],"2–19",[310],"CSA2, compression m = 2",[312],"3 groups of 6: 1 Full + 5 Reuse",[314,316,318,320],[315],"Decoder",[317],"20–23",[319],"CSA2, m = 1",[321],"1 Full + 3 Reuse",[323,324,326,327],[315],[325],"24–39",[319],[328],"4 groups of 4: 1 Reindex + 3 Reuse",{"type":127,"content":330},[331,332,335,336,339],"Count it out: of 38 layers with global attention, only ",{"tag":131,"children":333},[334],"4"," compute their own main KV, 4 more re-run an indexer over a shared cache, and ",{"tag":131,"children":337},[338],"30"," just reuse. Note that the decoder's single Full layer uses compression m = 1, so the decoder reads uncompressed entries, while the encoder caches at half resolution. CSA2 also simplifies the compressor: V4's CSA built each compressed entry from 2m overlapping source entries with an absolute positional embedding; CSA2 drops the overlap and the positional embedding, and derives indexer keys by projecting the main KV instead of running a separate compression path. Fewer moving parts, faster training.",{"type":159,"level":341,"text":342},3,"The hierarchical sparse indexer",{"type":127,"content":344},[345],"Reuse cuts the number of indexer passes, but every remaining indexer still scores the entire visible context, which grows linearly with length. In the decoder, the first Full layer therefore does double duty: besides its own top-512, it scores blocks of 8 positions by their maximum index score and keeps the best 2,048 blocks, a candidate pool of 16,384 positions. The later Reindex layers only score inside that pool. For a fixed pool size their per-query cost becomes constant instead of linear in context length. It is introduced in post-training and applied identically in training and inference, so the deeper indexers learn under the same restriction they serve with.",{"type":159,"level":160,"id":99,"text":100},{"type":127,"content":348},[349,350,353],"The third lever is precision. V4 already used FP4 for indexer queries and keys; V4.1 Flash extends quantization-aware training to the ",{"tag":131,"children":351},[352],"main KV cache",". The format is E2M1 values with one E4M3 scale per 16 channels, which is NVFP4 without its second-level global scale. Dropping the global scale is safe here because the numbers are bounded: after RMSNorm and RoPE the 512-channel KV latent cannot exceed about √512 ≈ 22.6 per channel, the observed maximum during training was around 10, and the format reaches 448 × 6 = 2,688. The cache is quantized after RoPE. The sliding-window cache stays in FP8, because the paper found it too sensitive to quantization.",{"type":127,"content":355},[356,357,359,360,363],"Put the three levers together and the global cache, the part that has to stay in HBM, lands at ",{"tag":131,"children":358},[208],": about a quarter of V4 Flash and, by DeepSeek's chart, 437 times smaller than DeepSeek V1. At a full 1M-token context that is roughly 0.9 GB of global cache per sequence. On top of that, ",{"tag":131,"children":361},[362],"Encoder SWA Bounded Replay"," means sliding-window state is never written to the persistent prefix cache at all; on a cache hit, the model replays the last 128 tokens of the cached prefix to rebuild an approximate local state. That takes the persistent footprint on SSD or host memory to about one eighth of V4 Flash.",{"type":127,"content":365},[366],"Can the 890 be reconstructed? Roughly, under assumptions the paper does not spell out. If each main KV entry is the 512-channel latent at 4.5 bits (288 bytes) and each indexer key is 128 channels at the same rate (72 bytes), the decoder's one shared cache at m = 1 costs 360 bytes per token and the encoder's three Full-layer caches at m = 2 cost 3 × 180 = 540. That is about 900 bytes, close to the published figure. Treat this as my back-of-the-envelope check, not DeepSeek's breakdown; it is mainly useful to see that the number is dominated by how few layers own a cache.",{"type":127,"content":368},[369],"The compute side follows the same curve. The paper's Figure 2 shows that extending the context 256-fold, from 4K to 1M tokens, raises single-token decode FLOPs by only about a quarter. Decode cost is close to flat in context length, which is exactly what you want for a long-running agent.",{"type":159,"level":160,"id":102,"text":103},{"type":127,"content":372},[373],"Three further components are less central to the cache story but explain the parameter count and the serving speed:",{"type":265,"ordered":266,"items":375},[376,381,386],[377,380],{"tag":131,"children":378},[379],"Engram"," is a conditional memory module that separates memorisation from computation: hashed n-gram lookups (orders 2, 3 and 4, eight hash heads, tables of about 16M entries each) feed embeddings into the residual stream. Its 196B parameters sit in two modules at layers 1 and 14, stored in FP8. Because addressing is deterministic from the input tokens, the serving stack can prefetch the rows from host memory over RDMA while the first transformer block runs, so these parameters do not need GPU memory.",[382,385],{"tag":131,"children":383},[384],"DSpark"," replaces the multi-token prediction module of V3 with a separately trained speculative decoder: three transformer blocks with a 128-token window draft five positions in one pass, a small Markov head models dependencies between draft tokens, and a confidence head predicts acceptance so the scheduler can pick the verification length per request based on current load. It is trained after pre-training with the backbone frozen and kept in sync during RL, so it also speeds up rollout generation.",[387,390],{"tag":131,"children":388},[389],"Single-pass mHC"," tweaks the multi-stream residual connection from V4 (four residual streams). Each block now uses the input-mixing coefficients computed by the previous block, which removes a data dependency and lets one fused kernel read and write the residual once. The paper reports this halves activation memory traffic, from (4n + 4)·d to (2n + 2)·d, with negligible quality cost.",{"type":127,"content":392},[393],"Image input is native rather than bolted on. A DeepSeek-ViT encoder (32 layers, patch size 14, 2D-RoPE) feeds the backbone through a 3 × 3 pixel-unshuffle that cuts visual tokens by a factor of nine, supporting inputs up to about 1344 × 1344 pixels. Text and image tokens get separate load-balancing biases in the MoE router, so one modality cannot hide expert imbalance in the other.",{"type":159,"level":160,"id":105,"text":106},{"type":127,"content":396},[397],"Pre-training ran on 45T tokens with a 7 : 1 ratio of text-only to multimodal data, at a fixed batch of 100.6M tokens. Sparse attention was trained from scratch at 64K sequence length, with no dense warm-up stage, and extended to 1M tokens at the 34T mark. The learning rate held at 2.6 × 10⁻⁴ until 28T tokens, decayed on a cosine schedule to 2.6 × 10⁻⁵ by 40T, and stayed there to the end. The paper reports no instability.",{"type":127,"content":399},[400,401,404],"The optimiser setup is worth a look if you train models yourself. Weight matrices use Muon, with ",{"tag":131,"children":402},[403],"head-wise Muon"," for query and key weights so each attention head gets its own preconditioner. The large embedding-like matrices (Engram tables, token embedding, output head) skip Adam, whose optimiser state would be huge for 196B parameters, in favour of a momentum update followed by Sinkhorn balancing that equalises row and column RMS. DeepSeek reports this beats Adam while needing only a momentum buffer. The result, V4.1-Flash-Base, matches V4-Pro-Base on knowledge, reasoning and coding with a third of the total and a quarter of the active parameters.",{"type":159,"level":341,"text":406},"Reasoning effort as a trained dial",{"type":127,"content":408},[409,410,414],"The post-training recipe is deliberately boring: supervised fine-tuning, RL and on-policy distillation, as in V4, with the effort spent on synthesised agent tasks and environments. One mechanism matters for anyone deploying it. During RL the system prompt carries a line ",{"tag":411,"children":412},"code",[413],"Reasoning Effort: {effort}"," with a value from 1 to 100, and the reward includes a length penalty on reasoning tokens whose coefficient falls exponentially as the effort value rises. Rewards are normalised within each effort level, so the model learns a different length budget per level rather than one compromise.",{"type":127,"content":416},[417,418,421,422,425,426,429],"The public API exposes three presets that map onto that scalar: ",{"tag":131,"children":419},[420],"max"," = 100, ",{"tag":131,"children":423},[424],"high"," = 75, ",{"tag":131,"children":427},[428],"low"," = 50. If you self-host, you are not limited to the presets: the paper notes that intermediate values interpolate between trained levels.",{"type":159,"level":160,"id":108,"text":109},{"type":127,"content":432},[433,434,437,438,441],"1017 keeps the architecture and changes the policy. DeepSeek's ",{"tag":136,"href":54,"children":435},[436],"announcement"," calls it a Pareto improvement: better scores, faster task completion and shorter outputs at once, tuned for DeepSeek's own harness and third-party agents such as Tencent CodeBuddy. ",{"tag":136,"href":48,"children":439},[440],"The Decoder"," reports that the update uses reinforcement learning with self-feedback (RLSF) for building complete projects from scratch, where the model's own confidence or self-critique serves as the reward signal. The model card does not detail the method, so treat that as reported rather than documented.",{"type":127,"content":443},[444,445,448],"Here are the numbers from the ",{"tag":136,"href":42,"children":446},[447],"model card",", all reported by DeepSeek:",{"type":162,"head":450,"rows":464},[451,453,454,456,458,460,462],[452],"Benchmark",[133],[455],"V4.1 Flash",[457],"Kimi K3",[459],"GLM-5.3",[461],"GPT-5.6 Sol",[463],"Opus 5",[465,482,499,516,533,549,566,583,600,616,632],[466,468,470,472,474,476,478],[467],"HLE, no tools",[469],"42.6",[471],"36.8",[473],"43.5",[475],"42.0",[477],"44.5",[479],{"tag":131,"children":480},[481],"56.3",[483,485,487,489,491,493,495],[484],"Terminal-Bench 4.0",[486],"42.3",[488],"31.2",[490],"12.6",[492],"37.9",[494],"39.9",[496],{"tag":131,"children":497},[498],"51.8",[500,502,506,508,510,512,514],[501],"DeepSWE v1.1",[503],{"tag":131,"children":504},[505],"77.0",[507],"74.2",[509],"67.5",[511],"66.9",[513],"73.0",[515],"74.0",[517,519,521,523,525,527,529],[518],"ProgramBench",[520],"29.1",[522],"20.3",[524],"17.5",[526],"19.0",[528],"23.0",[530],{"tag":131,"children":531},[532],"37.0",[534,536,538,540,542,543,545],[535],"NL2Repo-Bench",[537],"68.2",[539],"65.4",[541],"58.0",[541],[544],"56.8",[546],{"tag":131,"children":547},[548],"75.3",[550,552,556,558,560,562,564],[551],"Automation-Bench v1.0.6",[553],{"tag":131,"children":554},[555],"60.5",[557],"54.8",[559],"46.7",[561],"48.8",[563],"45.8",[565],"50.3",[567,569,573,575,577,579,581],[568],"Agents' Last Exam",[570],{"tag":131,"children":571},[572],"35.4",[574],"31.8",[576],"27.6",[578],"28.5",[580],"26.7",[582],"28.6",[584,586,590,592,594,596,598],[585],"Apex Bench v1.1",[587],{"tag":131,"children":588},[589],"72.1",[591],"64.6",[593],"47.8",[595],"56.6",[597],"54.3",[599],"65.8",[601,603,605,607,609,610,612],[602],"Chartography (tools)",[604],"83.0",[606],"78.9",[608],"68.1",[304],[611],"79.9",[613],{"tag":131,"children":614},[615],"84.0",[617,619,621,623,625,626,628],[618],"BabyVision (tools)",[620],"92.5",[622],"89.6",[624],"85.7",[304],[627],"88.9",[629],{"tag":131,"children":630},[631],"94.1",[633,635,637,639,641,642,646],[634],"ZeroBench-main (tools)",[636],"51.0",[638],"49.0",[640],"41.0",[304],[643],{"tag":131,"children":644},[645],"53.0",[647],"52.0",{"type":127,"content":649},[650],"Read the table in two directions. Against its predecessor, 1017 improves on every row, with the largest jumps where agents work in a terminal or build whole programs: Terminal-Bench 4.0 +11.1, ProgramBench +8.8, Apex +7.5, HLE +5.8. Against Opus 5, the picture is split. Flash wins on workflow-style agent benchmarks (DeepSWE, Automation-Bench, Agents' Last Exam, Apex) and loses clearly on hard reasoning and deep engineering: 14 points on HLE, 9.5 on Terminal-Bench 4.0, 8 on ProgramBench, 7 on NL2Repo. The technical report says the same in plain words: a gap with giant models remains on science-oriented agentic tasks that need expert domain knowledge.",{"type":152,"variant":652,"title":653,"body":654},"warn","Vendor numbers, vendor harnesses",[655],[656],"Every score above comes from DeepSeek, run with harnesses DeepSeek chose (Claude Code for Terminal-Bench 4.0, Mini-SWE-V2 for DeepSWE). Independent results were not out at the time of writing. A benchmark is a shortlist, not a decision: run your own eval set before you route production traffic.",{"type":159,"level":160,"id":111,"text":112},{"type":127,"content":659},[660,661,664],"Shorter outputs are where the update pays off. DeepSeek's cost chart for DeepSWE, as reported by The Decoder, puts 1017 at 77.0% for just under ",{"tag":131,"children":662},[663],"$0.20 per task",". At the low effort setting it still clears 72% for about $0.10. The previous V4.1 Flash scored 74.2% at around $0.50. The same chart places GPT-6 Astra, Gemini 3.8 Flash and Claude Opus 5 between $2 and more than $10 per task. Same caveat as above: this is the vendor's chart, and accuracy per dollar on your tasks may look different.",{"type":127,"content":666},[667,668,670,671,674],"The list price on ",{"tag":136,"href":45,"children":669},[138]," is $0.15 per million input tokens, $0.60 per million output tokens and ",{"tag":131,"children":672},[673],"$0.003 per million cache reads",", with a 1M context and up to 393,216 output tokens. There is a detail most write-ups miss: the prices are time-of-day based. On weekdays from 01:00 to 04:00 UTC and from 06:00 to 10:00 UTC, which is the Beijing working day, all three rates double. Batch jobs scheduled outside those windows pay half.",{"type":127,"content":676},[677,678,681,682,686],"The ratio that matters is cache read versus fresh input: 50 to 1. Take an illustrative agent session of 50 turns, each with a 200K-token context of which 195K is a cached prefix, 5K new tokens and 2K tokens of output. Per turn that is about $0.0006 for cache reads, $0.00075 for fresh input and $0.0012 for output, so roughly ",{"tag":131,"children":679},[680],"$0.13 for the whole session"," off-peak. Break the prefix so nothing is cached, and the input alone costs $0.03 per turn, $1.50 per session: more than ten times as much for the same work. The architecture made the cache cheap to keep; your prompt structure decides whether you get to use it. My ",{"tag":224,"to":683,"children":684},"\u002Fblog\u002Fllm-cost-latency-prompt-caching-routing",[685],"prompt caching and routing guide"," covers how to keep the prefix stable.",{"type":127,"content":688},[689,690,693],"Alongside the model, DeepSeek opened its in-house ",{"tag":131,"children":691},[692],"Search and Fetch APIs"," at $0.75 per 1,000 successful requests, capped at 5 requests per second per account. Calls made as server-side tools through DeepSeek Harness, Claude Code or Codex are free until the end of October 2026.",{"type":159,"level":160,"id":114,"text":115},{"type":127,"content":696},[697,698,701,702,705,706,709,710,713,714,717],"Through OpenRouter the model is a normal OpenAI-compatible chat completion. The model card recommends ",{"tag":411,"children":699},[700],"temperature"," and ",{"tag":411,"children":703},[704],"top_p"," of 1.0 and a ",{"tag":411,"children":707},[708],"max_tokens"," budget of at least 256K for long agent runs; the effort preset goes through OpenRouter's ",{"tag":411,"children":711},[712],"reasoning"," parameter. On DeepSeek's own API the model is served as ",{"tag":411,"children":715},[716],"deepseek-flash",".",{"type":411,"code":719},"\u002F\u002F Node 22+, OpenRouter. Low effort for routine agent turns, high or max for hard ones.\nconst res = await fetch('https:\u002F\u002Fopenrouter.ai\u002Fapi\u002Fv1\u002Fchat\u002Fcompletions', {\n  method: 'POST',\n  headers: {\n    Authorization: `Bearer ${process.env.OPENROUTER_API_KEY}`,\n    'Content-Type': 'application\u002Fjson',\n  },\n  body: JSON.stringify({\n    model: 'deepseek\u002Fdeepseek-v4.1-flash-1017',\n    reasoning: { effort: 'low' },\n    temperature: 1.0,\n    top_p: 1.0,\n    messages: [\n      \u002F\u002F Keep this prefix byte-identical across turns: cache reads cost 1\u002F50 of fresh input.\n      { role: 'system', content: SYSTEM_PROMPT },\n      ...history,\n      { role: 'user', content: task },\n    ],\n  }),\n})\nconst data = await res.json()\nconsole.log(data.choices[0].message.content, data.usage)",{"type":127,"content":721},[722,723,727,728,732],"Self-hosting is possible, and the MIT license makes it legally simple, but it is not a single-GPU project: count on around 750B parameters including the Engram tables, plus an inference stack that understands CED, CSA2, FP4 KV and bounded replay. The weights repository ships its own inference code and conversion instructions; engines such as ",{"tag":224,"to":724,"children":725},"\u002Ftools\u002Fvllm",[726],"vLLM"," usually need some time to support a new attention design efficiently. If you go this way for data-protection reasons, the trade-offs in ",{"tag":224,"to":729,"children":730},"\u002Fblog\u002Fself-hosted-llm-gdpr-cost",[731],"self-hosted LLMs, GDPR and cost"," apply directly. With your own deployment you can also set the effort value directly in the system prompt instead of choosing a preset:",{"type":411,"code":734},"Reasoning Effort: 60 (range 1–100; higher values request more thorough reasoning)",{"type":159,"level":160,"id":117,"text":118},{"type":265,"ordered":266,"items":737},[738,743,748],[739,742],{"tag":131,"children":740},[741],"Approximations by design."," Bounded replay reconstructs sliding-window state approximately, and the sparse indexer can pick the wrong 512 entries. DeepSeek's own conclusion names both as possible sources of degradation in untested edge cases, especially sparse retrieval over long contexts and state at cache-resumption boundaries.",[744,747],{"tag":131,"children":745},[746],"The frontier gap is real."," On hard reasoning and expert-level terminal work, Opus 5 stays well ahead. Flash is a strong default for volume, not a replacement for the strongest model on the hardest tasks.",[749,752,753,755],{"tag":131,"children":750},[751],"Jurisdiction."," The DeepSeek API is operated from China. The day before this release, ",{"tag":136,"href":48,"children":754},[440]," also reported that China plans to tighten rules on publishing open-weight models, which could affect future releases, though not weights already published under MIT.",{"type":159,"level":160,"id":120,"text":121},{"type":265,"ordered":758,"items":759},true,[760,765,770,778],[761,764],{"tag":131,"children":762},[763],"Put it on the shortlist for high-volume agent work",": test generation, refactors, code review passes, data extraction. That is where the cost per task gap is largest and the quality gap smallest.",[766,769],{"tag":131,"children":767},[768],"Run your own evals at low and high effort"," before switching anything. The low preset at roughly half the cost is the interesting operating point.",[771,774,775,717],{"tag":131,"children":772},[773],"Route by difficulty."," Keep a frontier model for the hard tail and let Flash handle the bulk, the same pattern as in my ",{"tag":224,"to":683,"children":776},[777],"routing guide",[779,782],{"tag":131,"children":780},[781],"Engineer for the cache and the clock."," Stable prefixes and off-peak batch scheduling are worth more than any benchmark point at these prices.",{"type":127,"content":784},[785,786,790],"The broader lesson from this release is that the next price drops will come from memory, not from FLOPs. A model that needs a quarter of the cache can serve four times the concurrent long-context sessions on the same hardware, and that is what makes $0.20 agent tasks possible. If you want help working out where a model like this fits into a real system, the ",{"tag":224,"to":787,"children":788},"\u002Fexpertise\u002Fai-engineer",[789],"AI engineering"," page describes how I work.",{"type":159,"level":160,"id":123,"text":124},{"type":265,"ordered":266,"items":793},[794,799,804,809,814,819],[795,798],{"tag":136,"href":39,"children":796},[797],"DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression",", technical report, arXiv:2609.19969",[800,803],{"tag":136,"href":42,"children":801},[802],"DeepSeek-V4.1-Flash-1017 model card",", Hugging Face",[805,808],{"tag":136,"href":45,"children":806},[807],"DeepSeek V4.1 Flash 1017 pricing and providers",", OpenRouter (prices as of 17 October 2026)",[810,813],{"tag":136,"href":48,"children":811},[812],"Deepseek V4.1 Flash 1017 beats larger rivals in agentic tasks",", The Decoder",[815,818],{"tag":136,"href":51,"children":816},[817],"DeepSeek opens V4.1-Flash-1017 weights and $0.75 Search API",", AI Weekly",[820,823],{"tag":136,"href":54,"children":821},[822],"DeepSeek launch announcement",", X",[825,893,954,1010],{"slug":826,"published":5,"minutes":827,"category":7,"tags":828,"keywords":834,"about":846,"sources":856,"cover":887,"og":888,"expertise":57,"locales":889,"lang":59,"title":890,"description":891,"coverAlt":892},"llm-rate-limits-retries-provider-fallback",10,[829,830,831,832,833],"Rate limits","Retries","Fallback","Reliability","LLM gateway",[835,836,837,838,839,840,841,842,843,844,845],"llm rate limit handling","429 too many requests llm api","exponential backoff with jitter","anthropic 529 overloaded error","openai rate limit retry","llm provider fallback","litellm fallbacks","retry-after header","llm circuit breaker","multi provider llm failover","claude api rate limits",[847,850,853],{"name":848,"url":849},"Token bucket","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FToken_bucket",{"name":851,"url":852},"Exponential backoff","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FExponential_backoff",{"name":854,"url":855},"Circuit breaker design pattern","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FCircuit_breaker_design_pattern",[857,860,863,866,869,872,875,878,881,884],{"title":858,"url":859},"Claude Platform Docs: Rate limits","https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fapi\u002Frate-limits",{"title":861,"url":862},"Claude Platform Docs: API errors","https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fapi\u002Ferrors",{"title":864,"url":865},"OpenAI API Docs: Rate limits","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fguides\u002Frate-limits",{"title":867,"url":868},"AWS Builders Library: Timeouts, retries and backoff with jitter","https:\u002F\u002Fbuilder.aws.com\u002Fcontent\u002F3EumjoZascWd1oZiEgL8ORlv3qE\u002Ftimeouts-retries-and-backoff-with-jitter",{"title":870,"url":871},"MDN: Retry-After header","https:\u002F\u002Fdeveloper.mozilla.org\u002Fen-US\u002Fdocs\u002FWeb\u002FHTTP\u002FReference\u002FHeaders\u002FRetry-After",{"title":873,"url":874},"Martin Fowler: Circuit Breaker (2014)","https:\u002F\u002Fmartinfowler.com\u002Fbliki\u002FCircuitBreaker.html",{"title":876,"url":877},"LiteLLM Docs: Router, retries and cooldowns","https:\u002F\u002Fdocs.litellm.ai\u002Fdocs\u002Frouting",{"title":879,"url":880},"LiteLLM Docs: Proxy fallbacks and reliability","https:\u002F\u002Fdocs.litellm.ai\u002Fdocs\u002Fproxy\u002Freliability",{"title":882,"url":883},"OpenRouter Docs: Model fallbacks","https:\u002F\u002Fopenrouter.ai\u002Fdocs\u002Fguides\u002Frouting\u002Fmodel-fallbacks",{"title":885,"url":886},"AI SDK Docs: Settings (maxRetries, timeout)","https:\u002F\u002Fai-sdk.dev\u002Fdocs\u002Fai-sdk-core\u002Fsettings","\u002Fimages\u002Fblog\u002Fllm-rate-limits-retries-provider-fallback\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fllm-rate-limits-retries-provider-fallback\u002Fog.jpg",[59,60,61],"LLM rate limits in production: retries, backoff and provider fallback","How LLM rate limits work, which errors to retry, backoff with jitter, retrying in one layer only, and fallback chains that do not break your output.","Small squares climbing three steps up to a wide rectangle, two more rectangles behind it in a row.",{"slug":894,"published":5,"minutes":895,"category":7,"tags":896,"keywords":902,"about":913,"sources":923,"cover":948,"og":949,"expertise":57,"locales":950,"lang":59,"title":951,"description":952,"coverAlt":953},"prompt-versioning-management",8,[897,898,899,900,901],"Prompt management","Versioning","LLMOps","Evals","Langfuse",[903,904,905,906,907,908,909,910,911,912],"prompt versioning","prompt management","prompt registry","version control for prompts","langfuse prompt management","mlflow prompt registry","prompts in git","openai prompt object deprecation","prompt rollback production","prompt ci evaluation",[914,917,920],{"name":915,"url":916},"Prompt engineering","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FPrompt_engineering",{"name":918,"url":919},"Version control","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FVersion_control",{"name":921,"url":922},"Continuous integration","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FContinuous_integration",[924,927,930,933,936,939,942,945],{"title":925,"url":926},"OpenAI API Docs: Migrate from prompt objects","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fguides\u002Fprompting\u002Fmigrate-from-prompt-object",{"title":928,"url":929},"OpenAI API Docs: Prompt engineering (reusable prompts deprecation note)","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fguides\u002Fprompt-engineering",{"title":931,"url":932},"Langfuse Docs: Prompt version control","https:\u002F\u002Flangfuse.com\u002Fdocs\u002Fprompt-management\u002Ffeatures\u002Fprompt-version-control",{"title":934,"url":935},"Langfuse Docs: Prompt caching in the SDKs","https:\u002F\u002Flangfuse.com\u002Fdocs\u002Fprompt-management\u002Ffeatures\u002Fcaching",{"title":937,"url":938},"Langfuse Docs: Link prompts to traces","https:\u002F\u002Flangfuse.com\u002Fdocs\u002Fprompt-management\u002Ffeatures\u002Flink-to-traces",{"title":940,"url":941},"MLflow Docs: Prompt Registry","https:\u002F\u002Fmlflow.org\u002Fdocs\u002Flatest\u002Fgenai\u002Fprompt-registry\u002F",{"title":943,"url":944},"Promptfoo Docs: GitHub Action","https:\u002F\u002Fwww.promptfoo.dev\u002Fdocs\u002Fintegrations\u002Fgithub-action\u002F",{"title":946,"url":947},"Claude Platform Docs: Prompt caching","https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fbuild-with-claude\u002Fprompt-caching","\u002Fimages\u002Fblog\u002Fprompt-versioning-management\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fprompt-versioning-management\u002Fog.jpg",[59,60,61],"Prompt versioning in production: git, registries, labels and rollbacks","How to version prompts in production: what belongs in a version, git versus a prompt registry, labels and rollbacks, eval gates in CI and cache side effects.","Five upright cards in a row, the third turned back on a curved arrow.",{"slug":955,"published":5,"minutes":956,"category":7,"tags":957,"keywords":962,"about":973,"sources":983,"cover":1004,"og":1005,"expertise":57,"locales":1006,"lang":59,"title":1007,"description":1008,"coverAlt":1009},"llm-batch-api-offline-workloads",9,[958,959,899,960,961],"Batch API","Cost","Prompt caching","Data pipelines",[963,964,965,966,967,968,969,970,971,972],"llm batch api","anthropic message batches api","openai batch api","gemini batch mode","batch inference llm cost","batch api 50 percent discount","openai flex processing","bedrock batch inference","llm offline processing","batch api prompt caching",[974,977,980],{"name":975,"url":976},"Batch processing","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FBatch_processing",{"name":978,"url":979},"JSON Lines","https:\u002F\u002Fjsonlines.org\u002F",{"name":981,"url":982},"Large language model","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FLarge_language_model",[984,987,989,990,993,996,998,1001],{"title":985,"url":986},"Claude Platform Docs: Batch processing","https:\u002F\u002Fplatform.claude.com\u002Fdocs\u002Fen\u002Fbuild-with-claude\u002Fbatch-processing",{"title":988,"url":859},"Claude Platform Docs: Rate limits (Message Batches API)",{"title":946,"url":947},{"title":991,"url":992},"OpenAI API Docs: Batch API","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fguides\u002Fbatch",{"title":994,"url":995},"OpenAI API Docs: Flex processing","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fguides\u002Fflex-processing",{"title":997,"url":865},"OpenAI API Docs: Rate limits (batch queue limits)",{"title":999,"url":1000},"Google AI for Developers: Gemini Batch API","https:\u002F\u002Fai.google.dev\u002Fgemini-api\u002Fdocs\u002Fbatch-api",{"title":1002,"url":1003},"Amazon Bedrock User Guide: Batch inference","https:\u002F\u002Fdocs.aws.amazon.com\u002Fbedrock\u002Flatest\u002Fuserguide\u002Fbatch-inference.html","\u002Fimages\u002Fblog\u002Fllm-batch-api-offline-workloads\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fllm-batch-api-offline-workloads\u002Fog.jpg",[59,60,61],"LLM batch APIs: half-price inference for work that can wait","When to use the Anthropic, OpenAI and Gemini batch APIs, how limits, result matching and failures work, and how batch discounts stack with prompt caching.","A wide box holding eight small squares, tiny squares arriving from the left, one long arrow returning.",{"slug":1011,"published":5,"minutes":1012,"category":7,"tags":1013,"keywords":1018,"about":1030,"sources":1038,"cover":1075,"og":1076,"expertise":57,"locales":1077,"lang":59,"title":1078,"description":1079,"coverAlt":1080},"small-language-models-when-enough",11,[1014,1015,1016,1017,899],"Small language models","Distillation","Quantization","Model routing",[1019,1020,1021,1022,1023,1024,1025,1026,1027,1028,1029],"small language models","when to use a small language model","SLM vs LLM cost","model distillation","quantization aware training","Q4_K_M","LLM routing","small model evaluation","on-device LLM","Gemma 4 E4B","local LLM privacy",[1031,1034,1037],{"name":1032,"url":1033},"Small language model","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FSmall_language_model",{"name":1035,"url":1036},"Knowledge distillation","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FKnowledge_distillation",{"name":981,"url":982},[1039,1042,1045,1048,1051,1054,1057,1060,1063,1066,1069,1072],{"title":1040,"url":1041},"Belcak et al.: Small Language Models are the Future of Agentic AI (arXiv 2506.02153)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2506.02153",{"title":1043,"url":1044},"Google: Gemma 4 model card","https:\u002F\u002Fai.google.dev\u002Fgemma\u002Fdocs\u002Fcore\u002Fmodel_card_4",{"title":1046,"url":1047},"Google: Gemma 3 QAT models, state-of-the-art AI on consumer GPUs","https:\u002F\u002Fdevelopers.googleblog.com\u002Fen\u002Fgemma-3-quantized-aware-trained-state-of-the-art-ai-to-consumer-gpus\u002F",{"title":1049,"url":1050},"Google: Introducing Gemma 3 270M","https:\u002F\u002Fdevelopers.googleblog.com\u002Fen\u002Fintroducing-gemma-3-270m\u002F",{"title":1052,"url":1053},"Microsoft Research: Phi-4 Technical Report (arXiv 2412.08905)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2412.08905",{"title":1055,"url":1056},"Qwen: Qwen3 Technical Report (arXiv 2505.09388)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2505.09388",{"title":1058,"url":1059},"Apple: Apple Intelligence Foundation Language Models, Tech Report 2025 (arXiv 2507.13575)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2507.13575",{"title":1061,"url":1062},"Ong et al.: RouteLLM, Learning to Route LLMs with Preference Data (arXiv 2406.18665)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2406.18665",{"title":1064,"url":1065},"LMSYS: RouteLLM, an open-source framework for cost-effective LLM routing","https:\u002F\u002Flmsys.org\u002Fblog\u002F2024-07-01-routellm\u002F",{"title":1067,"url":1068},"Gomes et al.: On the Generalization vs Fidelity Paradox in Knowledge Distillation (arXiv 2505.15442)","https:\u002F\u002Farxiv.org\u002Fabs\u002F2505.15442",{"title":1070,"url":1071},"llama.cpp: quantize tool documentation","https:\u002F\u002Fgithub.com\u002Fggml-org\u002Fllama.cpp\u002Fblob\u002Fmaster\u002Ftools\u002Fquantize\u002FREADME.md",{"title":1073,"url":1074},"European Commission: AI Act","https:\u002F\u002Fdigital-strategy.ec.europa.eu\u002Fen\u002Fpolicies\u002Fregulatory-framework-ai","\u002Fimages\u002Fblog\u002Fsmall-language-models-when-enough\u002Fcover.webp","\u002Fimages\u002Fblog\u002Fsmall-language-models-when-enough\u002Fog.jpg",[59,60,61],"Small language models: when a 4B model is enough","When a small model beats a frontier API: cost per resolved case, latency, privacy, distillation, quantization, routing and an eval plan that fits your traffic.","A small cube, a fork in the path, a check on one branch and a larger hollow cube on the other.",1792245345350]