DeepSeek open-sources the engine that runs its AI models in production

Chinese AI company DeepSeek released its inference engine DeepInfer as open source under the Apache 2.0 license on October 20, 2026. Inference is the step where a fully trained AI model actually answers questions. According to The Next Web, DeepInfer is the engine DeepSeek itself uses to serve its models in production, and it sits behind the company's API, the access developers use to connect the models to their own systems.

According to The Next Web, the GitHub repository gained more than 8,000 stars within a few hours of the release.

Limited at launch

At launch, the engine supports only one model, DeepSeek-V4.1-Flash. According to The Next Web, more models are planned for later.

On the hardware side, two platforms are supported: Nvidia's SM100, which belongs to the Blackwell generation, and Huawei's Ascend 950 NPUs. Nvidia's older Hopper generation is not supported, VKTR reports.

VKTR also points out that the release leaves out parts of the production setup that larger users typically need: a KV cache system, separation of prefill and decode, and load balancing across replicas.

What's included

According to the README on GitHub, DeepInfer was originally built primarily on the well-known engine vLLM, with additional inspiration from SGLang and Nvidia's TensorRT-LLM. The engine includes optimized kernels (small, fast pieces of code for the heavy computations), speculative decoding, asynchronous scheduling and structured output, where the response can be forced into a specific format such as a JSON schema.

The benchmarks shown were measured on 4 SM100 GPUs for prefill and 64 GPUs for decode. The speed figures are DeepSeek's own and, according to The Next Web, have not been independently tested. VKTR writes that performance figures have been published only for Nvidia hardware, and that there are no public figures for running on Huawei.

What it could mean

DeepSeek says the company is sharing its production stack to help the community lower computing costs, reduce energy consumption and make AI cheaper. The Next Web assesses that the release could lower costs for organizations that run DeepSeek models.

VKTR is more cautious: According to the outlet, enterprise use requires significant engineering effort, and there are hardware limitations as well as risk because of US export controls surrounding Huawei's Ascend chips.