DeepSeek open-sources DeepInfer, the engine behind its production serving

DeepSeek has open-sourced DeepInfer, the inference framework that powers its large-scale production services. According to the official repository, this release provides the offline inference build with the core features needed to serve models at scale.

The only supported model for now is DeepSeek-V4.1-Flash, the flagship the company has recently been serving. The framework is published under the Apache 2.0 license in the deepseek-ai organisation on GitHub, so outside teams can build and modify it directly.

The feature list is unusually concrete: optimized kernels (FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect and custom TileLang kernels), attention DP with balanced CP, MoE expert parallelism plus EPLB load balancing, and DSpark with confidence-scheduled verification for decoding.

There is more underneath: SWA bounded replay to cut decoder prefill work, asynchronous scheduling that overlaps CPU batch scheduling with device execution, batch-invariant inference so results stay identical across batch compositions, deterministic sampling, and structured outputs constrained by regex, JSON schema or EBNF grammar.

Hardware support carries the strongest signal. Both Nvidia GPUs and Huawei Ascend NPUs are listed. For a provider trying to push down cost per token, being able to draw on two supply chains means deployment flexibility and leverage on pricing.

Open source here does not mean a complete serving stack. The repository explicitly names what is not included: a KV cache system, VE/Prefill/Decode disaggregation (most engine functionality is implemented), and load balancing across data-parallel replicas. Put differently, the engine ships; some of the rest of the car does not.

That matches DeepSeek's broader pattern of releasing inference-layer engineering so more teams can run its models on their own hardware and the cost curve keeps bending downward. Outside coverage has read the release chiefly as a move to cut AI operating costs, compute and energy use.

What to watch next: whether the community fills in the missing serving components, how mature the Ascend path proves in practice, and whether the framework becomes a general inference substrate that other vendors' models can also borrow.