Performance¶
Measured behaviour, reported separately for prefill and decode. Every page here is
a run record: hardware, checkpoint, commit, and invocation are stated with the
numbers. Read Benchmarking and reporting rules first
if you intend to compare any of them, and do not carry a decode figure to a
different generation length — a TG4 measurement says nothing about TG512.
| Document | What it records |
|---|---|
| DSpark speculative decoding with adaptive draft-length gating | The DeepSeek-V4-Flash draft module, why the fixed accept threshold was falsified, and the online-calibrated gating that replaced it. |
| FlashMemory 1M context | The FlashMemory + KV_SWAP design for long-context memory reduction, and its current implementation status. (Chinese) |
| MiniMax-M2 decode bottleneck analysis | Per-phase profiling of MiniMax-M2 decode and prefill on TP4, and which phases the optimizations then targeted. |
| Qwen4-Exp heterogeneous TP4 performance | Moving Qwen4-Exp experts to host memory: measured prefill gain, and why the real floor was the disk rather than PCIe. |
| CPU offload and prefetch: the measured ceiling | Layer-granularity offload measured rather than planned: the H2D and overlap ceilings, the storage rate behind them, and why #156's decode and prefetch targets are unreachable on this host. |
| Qwen quantized KV cache at 65K (TG512) | The authoritative quantized-KV table at 65K/TG512, including corrections to two earlier throughput claims. |
| Qwen drafter acceptance and speedup | The three-way head-to-head of MTP, DSpark and DFlash2 on the Qwen GPU-resident FP8 path: MTP's 1.23x wall against the two width-8 drafters that lose to plain decode, and the per-row verify arithmetic that says why. |
| Native C++ OpenAI concurrency validation | The end-to-end HTTP acceptance test and the vLLM head-to-head that followed it. |
| Native C++ OpenAI tool-calling acceptance | Tool calling driven through a second turn, the openai SDK and langchain-openai, and the sidecar templating defect the first real two-turn request exposed. |
| Ascend 910A attention and its measured ceilings | The Cube (Mmad) GQA attention operator, the three-way decode dispatch, the prefill phase table, and the bandwidth, all-reduce and aclrtMemcpy limits every remaining target has to clear. |
| Ascend 910A TP collective overlap | Why the overlapped all-reduce must slice by row count rather than a constant, the crossover sweep, and a device-synced profile at three prompt lengths showing the linear-attention recurrence is 45-56% of the stack against the collectives' 13-25%. |
| Ascend decode: the all-reduce A/B, the event probe, and batch scaling | Why the decode all-reduce cannot be hidden on a second stream — seven measured orderings and the event probe that clears the primitive — and the batch table that reaches 114.5 TPS at TP4. |
| Ascend gated-delta value-axis slice | The measurement that shows the prefill recurrence is issue-bound rather than element-bound, why the cut is two slices and not four, the op-level and end-to-end A/B, and the run-to-run logit instability that rules the generated tokens out as evidence. |
| DeepSeek-V4 PersistentEngine serial baseline | The serial prefill/decode baseline for the native engine on DeepSeek-V4-Flash-0731, the environment ablation behind it, and why --max-batch-size 8 currently changes nothing. |
| Auditing the DeepSeek-V4.1-Flash shards from arrival to complete | A header audit that distinguishes a missing shard from a wrong one, run at four points of the download: 44.0% of the checkpoint readable at 20 shards, four Engram checks still undecided at 46, and 39 of 39 passed with nothing undecided on all 48 — plus the tensor shapes and per-layer counts the shards assert. |
| DeepSeek-V4.1-Flash: what the released checkpoint costs to run on one host | The byte census over all 96,085 tensors, the load report, what the tree occupies at TP4, the Engram gather's cold/warm/resident costs, what a generated token costs and what it is made of — 15 to 42 s on the host, 99.7% of it the fp4-to-bf16 expansion — and the measured PCIe and card-side rates that make the four cards the cheaper half of this model. |
| DeepSeek-V4.1-Flash: the routed experts on the four cards | The expert-parallel split over four 2080 Ti: the static 2/2/1/1 deal, the fp4 kernel's parity against the host expert, a measured 1.06–1.14 s per decode step phase by phase — and the two honest negative results behind it: the four cards are worth 0.1–0.2 s of that, because staging is host work and the largest term, while the launch was four kernels serialized rather than one plus copies, worth 3.4× once the drain moved out of the card loop. |
The currently authoritative per-model numbers live in the model guides; pages under Archive are historical records whose numbers have been superseded.