DeepSeek V4.1 Flash 1017 for Flare

This repository publishes signed Flare runtime packages for DeepSeek V4.1 Flash 1017, prepared for four-GPU tensor-parallel serving. Flare is a native Rust inference runtime. Each package revision contains prepared weights, compiled kernels, captured execution graphs and signed package metadata. Transformers, vLLM, SGLang and llama.cpp cannot load this format.

The model handles text and images, accepts up to 1,048,576 tokens per request, and serves an OpenAI-compatible API under the model name DeepSeek-V4.1-Flash.

About Flare

Flare is NVIDIA's open-source inference runtime for local DGX and GPU systems. It is written in Rust and serves prepared, signed packages directly on the GPU using precompiled kernels and captured execution graphs, with no Python and no runtime compilation. It supports tensor-parallel serving across multiple GPUs or hosts through Orbit, verifies publisher signatures before loading, and exposes an OpenAI-compatible API. The same packages run from a container image or from the flare executable installed through the signed APT repository.

The weights

The packages are built from deepseek-ai/DeepSeek-V4.1-Flash-1017, a mixture-of-experts text and image model released under the MIT license. The checkpoint has 40 layers, hidden width 5,120, 64 attention heads, 384 routed experts plus one shared expert, top-6 routing, 129,280 vocabulary entries, a 1,048,576-token context window, and three MTP draft layers. It stores MXFP4 expert weights with 32×32 blocks and E8M0 scales.

Flare prepares the checkpoint for each platform:

• gb202 keeps the checkpoint's losslessly compressed CSF expert scales.
• gb10 expands those scales to native E8M0 arrays, so serving does not run a scale-expansion kernel.

Every revision is a signed, self-contained TP4 package: a model.safetensors metadata primary, content-addressed companion files, and publisher.json. Keep the primary, every companion and publisher.json together under their original names. Do not mix files across revisions.

Pick a platform

The main revision holds this model card, the license and the publisher profile. Download the revision for your hardware; its README covers installation, launch, storage and troubleshooting for that platform.

Flare fundamentals

Install Flare

Use one of the two supported installations:

• Ubuntu or DGX OS: install the signed flare-inference APT package, which provides /usr/bin/flare. The gb10 setup shows the repository, key and fingerprint.
• Container: run ghcr.io/local-inference-lab/flare:release with the NVIDIA Container Toolkit. The gb202 setup shows the mounts and flags.

Source, guides and the Orbit multi-host reference live in NVIDIA/flare-local-inference.

Download and verify

Download with the Hugging Face hf CLI or with flare download, always naming an explicit revision. Flare verifies the package signature against publisher.json, then asks you to accept the publisher and the model license. The signature covers the metadata primary and every companion, so a missing or renamed payload fails verification. The writable trust store preserves license acceptance between launches.

Serve

The gb202 and gb10 packages are both TP4 and need four GPUs. Rank zero serves the OpenAI-compatible API at /v1, with health at /health; clients use the model name DeepSeek-V4.1-Flash.

The prepared packages share these capabilities:

• 32 concurrent requests and a 1,048,576-token per-request limit, subject to available KV memory.
• 4,096-token prefill chunks and adaptive MTP verification.
• Text and images through OpenAI image_url items (PNG, JPEG or static WebP), with four simultaneous image requests and a 4,096-visual-token budget per request.
• Sampling defaults of temperature 1.0, top-p 1.0, top-k disabled, min-p 0, repetition penalty 1.0, and zero presence/frequency penalties. Request values override them.
• Engram embedding tables held in system RAM or read from local disk, depending on the package and launch flags. Each platform README covers the recommended mode and memory budget.

Concurrency, prefill chunk size and draft-token layout are properties of the prepared package. Changing them requires a different package.

Documentation

• gb202 setup: x86-64, four RTX PRO 6000 Blackwell GPUs, Docker.
• gb10 setup: four DGX Sparks, APT install, agents and Orbit.
• Flare multi-host guide and Orbit reference.

Comprehensive model-accuracy evaluation has not been run.