PCIe GPUs Are Underrated: Kernel Fixes and Communication Rework Lift DeepSeek Inference Throughput Nearly 7x

METASTONE, a Chinese full-stack compute operator, says its in-house Meta-Infer deployment engine lifted input throughput for DeepSeek-V4.1-Flash from a community Day 0 baseline of 1,932 tokens per second to 13,274 tokens per second on a single machine with eight PCIe-only GPUs, a gain of roughly 6.87 times, while supporting a one-million-token context. The company says the improvement came from software alone, without changing model architecture or task semantics.

The backdrop is rising procurement costs and constrained supply of accelerator cards. As QbitAI reported on 24 September, competition in AI inference is shifting from who can buy the fastest hardware to who can extract more value from the compute already in hand. Plenty of GPU cards can load a model, download weights and start a service under mainstream open-source frameworks, so they “run”, yet benchmark throughput often falls far short of advertised levels. The model is not defective and the hardware works; the loss comes from a performance gap between framework and silicon.

Such cards typically lack high-speed interconnect between GPUs and sit outside the officially validated matrix of mainstream frameworks. Deploying the community default therefore triggers a cascade of problems: operators silently fall back to slow generic implementations, collective communication keeps tuning parameters written for NVLink hardware, and memory and parallelism settings inherit defaults from other hardware. Meta-Infer attacks this in two stages - co-designing model and hardware to unlock high-performance paths hidden by hardware differences, and rebuilding computation and communication to clear bottlenecks - resulting in four core capabilities: kernel completion, operator optimisation and communication rework, parallelism and capacity tuning, and cache reuse.

Kernel completion targets frameworks that do not raise errors but quietly route around the fast path. Many high-performance kernels are not missing at all; they fail to trigger because metadata, page sizes or hardware kernel bindings do not match. Meta-Infer aligns metadata definitions between hardware and framework, corrects page size configuration to unlock native fast operator paths, replaces legacy kernels unsuited to the target hardware, and widens the threshold at which fast communication paths take effect. On the eight-card PCIe-only machine, throughput for DeepSeek-V4.1-Flash rose from 1,932 to 5,850 tokens per second after adding the sparse-MLA prefill fast path, replacing a slow FP8 dense GEMM kernel and expanding PCIe-IPC fast-path coverage.

The same approach carried over to DeepSeek-V4-Flash, GLM5.3 and other models: correcting redundant computation in attention head padding, vectorising KV-cache marking logic, restructuring operator splitting for the hardware and extending CUDA graph coverage. Model-hardware co-design alone delivered throughput gains of 20 to 33 percent. Beyond that, the bottleneck moves to communication overhead, parallel scheduling, memory allocation and repeated computation. Meta-Infer fuses key operators such as attention and projection, overlaps computation with collective communication, runs statistical work inside communication gaps, and rewrites collective communication logic for PCIe bandwidth.

Parallelism and capacity tuning abandons globally fixed parallel parameters, configuring tensor and context parallelism differently for the prefill and decode phases, and adjusting static memory share and KV-cache capacity to avoid out-of-memory errors and operator fallback. Cache reuse targets long-text and video generation, judging when to reuse or refresh caches from live generation state without hurting output quality. On DeepSeek-V4.1-Flash, input throughput climbed from 5,850 to 13,274 tokens per second.

The methodology generalises. DeepSeek-V4-Flash rose from 14,546 to 22,584 input tokens per second, a 1.55x gain; GLM5.3 went from 3,236.78 to 6,222.72 tokens per second, 1.92x, with P95 time-to-first-token falling from 141.6 seconds to 46.6 seconds and supported context expanding from 270,000 to 1.05 million tokens. In video generation, MiniMax H3's 15-second reference-image-to-video task ran 2.48 times faster end to end, and a single eight-card machine reached 5.33 times baseline throughput for text-to-video and 4.98 times for reference-image-to-video.

The gap to top-tier hardware is narrower than intuition suggests. At matched concurrency points, a B300 delivered roughly 4.9 to 5.6 times the input throughput of this eight-card machine on DeepSeek-V4-Flash, and 3.5 to 4.7 times on GLM-5.3. In video generation at 5 seconds, 768p and 16:9, the machine produced 296 text-to-video clips per hour against 439 for the B300, and 131 reference-image-to-video clips against 225. Converted to whole-machine throughput, roughly 2.6 units of 6000D correspond to one B300 under the Ours Cache scheme, narrowing to about 1.5 and 1.7 units with Ours Cache plus LoRA.

The hardware itself did not change; the shrinking gap came from software stack, caching strategy and model optimisation combined. The real question is whether this purely software approach to bridging the hardware-framework divide will be opened up and ported to more long-tail accelerator cards and inference frameworks - for the many compute resources outside official validation matrices, that decides whether they merely run or are actually worth buying.