Spark-H3

Adaptive Block Sparse Attention for MiniMax-H3

Reblock similar tokens · Reweight tail tokens

Technical Blog · GitHub · ComfyUI · Documentation · Base Model

[!NOTE] This is a code-only release of the Spark-H3 attention backend. It does not redistribute MiniMax-H3 model weights. Download the base model from MiniMaxAI/MiniMax-H3.

Overview

Spark-H3 is the MiniMax-H3 implementation of Spark-Attn, a block sparse attention method for accelerating video generation while preserving output fidelity. It improves both branches of block-sparse attention:

• Spark-Reblock groups tokens with similar attention preferences so the exact branch spends its budget on more relevant interactions.
• Spark-Reweight builds weighted key/value summaries and corrects their attention mass, reducing the bias introduced by mean-pooled tail blocks.

Spark-H3 is an inference backend rather than a LoRA or a new checkpoint. It does not modify the MiniMax-H3 backbone weights and can be combined with community LoRAs.

Highlights

• Up to 2.33× attention speedup and 1.73× DiT speedup in a 10-second, 768p benchmark at 10% attention density. All sparse methods share a 20% dense warmup, with the first Transformer layer kept dense throughout.
• 23.30 dB PSNR at 10% attention density, compared with 20.36 dB for Sol-H3 under the same evaluation protocol.
• Spark-H3's Diffusers integration implements dedicated backends for NVIDIA SM80, SM89, and SM120 GPUs.
• Native ComfyUI nodes are implemented for SM89 and SM120, with prebuilt Linux x86_64 release packages available for both architectures.
• Thanks to ComfyUI's optimized MiniMax-H3 execution stack, Spark-H3 reaches up to 3.15× DiT speedup for 14.4-second, 1344×768 generation using the 20% attention-density Speed profile.

Unless otherwise noted, all latency measurements on this page were collected on a single NVIDIA RTX PRO 6000 Blackwell GPU.

Samples

ComfyUI — Dense, Fidelity, and Speed

Each video below contains three synchronized outputs from complete ComfyUI workflows: Dense with Spark-H3 disabled, Fidelity with 20% attention density and a short dense warmup, and Speed with 20% attention density and no dense warmup. Fidelity and Speed use dense_layers=0.

Across all cases, Native MiniMax-H3 uses 20 denoising steps and 4 Fidelity warmup steps. The LightX2V variant uses 8 denoising steps and 2 Fidelity warmup steps.

The prompts used in these comparisons were provided by Video DeltaNet (VDN) and LBH. We thank both teams for their awesome prompts.

1344×768 · 14.4 seconds · VDN Prompt 08

This case uses VDN Prompt 08, a cinematic Japanese city sequence.

Native MiniMax-H3

LightX2V 8-step v1.0 768p

1344×768 · 14.4 seconds · VDN Selected Prompt 06

This case shows a young woman opening an attic window into the rain.

Native MiniMax-H3

LightX2V 8-step v1.0 768p

1344×768 · 14.4 seconds · VDN Prompt 01

This case uses a cinematic Japanese city montage.

Native MiniMax-H3

LightX2V 8-step v1.0 768p

1920×1088 · 5.2 seconds · LBH prompt

This case uses an anime action sequence prompt from LBH.

Native MiniMax-H3

LightX2V 8-step v1.0 768p

VDN + LightX2V

The three samples below use VDN prompts 01, 06, and 10 from VDN blog with the LightX2V MiniMax-H3 Turbo LoRA and Spark-H3 at 10% attention density, with a 20% dense warmup and the first Transformer layer kept dense throughout.

VDN Prompt 01 — Japanese city montage

VDN Prompt 06 — Rainy-night camcorder documentary

VDN Prompt 10 — 2D anime fashion sequence

LBH two-stage sampling

The following samples combine Spark-H3 with SelfLift two-stage sampling based on the LBH learned latent upsampler.

LBH Case 01 — Black cat by the pool

LBH Case 02 — Stormtrooper on the beach

Ref2VA

Spark-H3 is also compatible with MiniMax-H3's Ref2VA pipeline, as shown on the official Ref2VA case.

Official Ref2VA case

See the technical blog for more results.

Benchmark Results

Evaluation setup. Dense attention, Sol-H3, Spark-H3, and Spark-H3-Lite are evaluated on VBench prompts with 19 denoising steps, 1344×768 resolution, and 240 frames at 24 fps (10 seconds). All sparse methods—Sol-H3, Spark-H3, and Spark-H3-Lite—use a 20% dense warmup (the first 4 of 19 Transformer evaluations) and keep the first Transformer layer dense throughout. Apart from these shared dense settings, Sol-H3 follows its official configuration. Quality metrics compare each method against dense outputs generated with the same prompts and seeds. Higher PSNR and SSIM are better; lower LPIPS is better.

At the 10% density operating point, Spark-H3 improves PSNR by 2.94 dB over Sol-H3 while also reducing DiT latency from 355.4 s to 337.4 s. Increasing the attention density provides a monotonic fidelity trade-off for standard Spark-H3, reaching 27.07 dB PSNR at 30% density.

Quick Start

Set up the upstream MiniMax-H3 pipeline first, then install Spark-H3 in the same environment with a compatible PyTorch/CUDA stack:

git clone https://github.com/zechengtang/Spark-H3.git
cd Spark-H3
python -m pip install -e '.[cuda]'

Wrap an already loaded MiniMax-H3 pipeline with the Spark installer:

from h3_sparse_attention import install_h3_spark_attn

num_denoise_steps = 19
num_inference_steps = num_denoise_steps + 1

with install_h3_spark_attn(
    pipe.transformer,
    num_denoise_steps=num_denoise_steps,
    warmup_mode="warmup_steps",
    warmup_steps=4,
    dense_layers=0,
    min_tokens=8192,
):
    result = pipe(**inputs, num_inference_steps=num_inference_steps)

MiniMax-H3's scheduler includes the terminal sigma value in num_inference_steps, but that point does not execute the Transformer. Pass the actual Transformer evaluation count to Spark as num_denoise_steps.

For configuration options, supported GPU architectures, and implementation details, see the Spark-H3 documentation.

ComfyUI

Spark-H3 provides released Linux ComfyUI nodes for NVIDIA SM89 and SM120 GPUs, along with example workflows. Follow the ComfyUI installation guide, then start with one of the workflows under workflows/.

License

Spark-H3's original code and documentation are released under the Apache License 2.0. Included third-party components retain their respective licenses and attribution notices. MiniMax-H3 model weights are not included and remain subject to the upstream MiniMax-H3 license.

Acknowledgments

Spark-H3 builds on MiniMax-H3 and Sol-Attn / Sol-Engine. The VDN prompt cases are sourced from the Video DeltaNet project.