DeepSeek Open Sources DeepInfer for Offline V4.1-Flash Inference on NVIDIA and Huawei Hardware

DeepSeek has published DeepInfer, the high-performance inference engine that powers its internal production services, as open-source software. According to SiliconANGLE, the release addresses the challenge of serving large models at scale. The framework is available under the Apache 2.0 license and focuses exclusively on offline inference capabilities for the DeepSeek-V4.1-Flash model.

What hardware and software does it support?

The framework runs on NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs. MarkTechPost reports that benchmarks were conducted using four GPUs for prefill and sixty-four for decode. It requires a Python 3.12 environment. The default checkpoint, DeepSeek-V4.1-Flash-1017, downloads from Hugging Face on first run unless a local path is specified via environment variables.

What features are missing from this release?

This version provides offline inference only. The GitHub README states that VE/Prefill/Decode disaggregation and load balancing across data-parallel replicas are required for a complete service but are not included. Most engine functionality for disaggregation is implemented, but the full orchestration layer is absent.

How does it handle performance and determinism?

DeepSeek uses optimized kernels including FlashMLA and DeepGEMM. Batch-invariant inference keeps results identical across batch compositions on NVIDIA hardware. Deterministic sampling produces identical outputs for identical inputs when configured correctly. DSpark accelerates decoding using confidence-scheduled verification with draft tokens.

Questions to ask your suppliers

Organizations should verify whether their current AI vendors are adopting similar open-source inference frameworks. Teams usually need to assess if offline-only capabilities meet their production latency requirements. Check whether suppliers provide clear documentation on hardware compatibility and performance benchmarks. Ensure that any integration plan accounts for the lack of built-in load balancing.

What to do and how to stay safe: DeepSeek

• Review your current inference stack for compatibility with NVIDIA SM100 or Huawei Ascend 950 hardware.
• Determine if offline inference capabilities meet your operational requirements before integrating new frameworks.
• Monitor the DeepSeek GitHub repository for updates on load balancing and disaggregation features.
• Evaluate the Apache 2.0 license terms for compliance with your organization's software usage policies.

Step-by-step guide: Browser Security Settings for Small Business Teams

General security guidance from the TechSec Feed newsroom. It is not confirmed advice from the organisations named in this story.

Frequently asked questions

Is DeepInfer available for online inference?

No, according to the source material, this release provides offline inference only.

Which license governs the DeepInfer framework?

DeepInfer is released under the Apache License 2.0.

Does it support load balancing?

No, load balancing across data-parallel replicas is not included in this release.