Sign up

Jiunsong/SuperGLM-5.3-Flash-NVFP4-DGX-Spark

🤗 On Hugging Faceimage-text-to-textmit165B params175 GBsafetensors✓ HF checksums availableupdated today
Magnet

SuperGLM-5.3-Flash-NVFP4-DGX-Spark

SuperGLM-5.3-Flash-NVFP4-DGX-Spark is a measured, attention-output-only derivative of the pinned ModelOpt NVFP4 GLM-5.3-Flash checkpoint. It is designed for two NVIDIA DGX Spark (GB10) nodes with the included SGLang runtime recipe. The release preserves the NVFP4 expert tensors and changes only the explicitly listed BF16 attention output projections.

What changed

  • Base: LibertAIDAI/GLM-5.3-Flash-NVFP4@aa28e1f54130286c95fee10d0705c74ce8743734
  • Donor: dealignai/GLM-5.3-Flash-UNCENSORED-NVFP4@d4d79fbbd474599db610b90a44b77497256ab518
  • Modified tensors: 30 × model.language_model.layers..self_attn.o_proj.weight
  • Selected layers: 15–44; blend 0.5; native MTP tensors modified by the intervention: false
  • Protected byte-for-byte: experts, routers, shared experts, mHC, KDA state/gates, DSA indexers, vision, embeddings, normalization, and lm_head

The intervention is derived from the independently compatible NVFP4 donor, while all non-selected tensor byte ranges are checked against the pristine base. The canonical OBLITERATUS registry contains 842 harmful/842 harmless pairs; prompts and outputs from the stress run are retained only as SHA-256 identities.

Architecture

Glm5NextForConditionalGeneration is a 45-layer hybrid with 11 DSA layers, 34 KDA layers, 288 routed experts, 8 experts/token, 4 mHC streams, RMSNorm, SiLU, and one native next-token prediction layer. ModelOpt NVFP4 uses group size 16 for routed-expert linear weights; attention, routing, shared experts, vision, embeddings, and heads remain BF16.

The config declares a 1,048,576-token maximum. This release only claims the directly served 16,384-token gate reported below.

Measured two-node DGX Spark results

Hardware: two NVIDIA GB10 nodes, TP=2, C1, RoCE v2 direct link. Throughput is the median of fixed-length streamed trials; TTFT includes prompt processing. The public numbers below come from independently reloaded release artifacts.

| Runtime | p256 decode tok/s | p256 TTFT s | p8192 decode tok/s | p8192 TTFT s | 16K |

|---|---:|---:|---:|---:|---|

| autoregressive | 14.79 | 0.558 | 14.64 | 8.499 | pass |

| native MTP | — | — | — | — | unsupported on measured 2×GB10 TP2 |

| DFlash2 | 23.90 | 0.602 | 28.17 | 9.885 | pass |

The autoregressive and DFlash2 reports passed chat/capability, tool calling, strict JSON structured output, low/high/max reasoning, Korean/Chinese/Japanese/French checks, and the bounded 16K context gate. The untouched baseline was measured first (pass).

The checkpoint retains its native MTP tensors, but native MTP is not claimed for the measured two-node GB10 TP2 topology. Independent rank-0/rank-1 startup attempts in the pinned GLM-5.3 support image failed while constructing the draft head with the same 4096-versus-2048 tensor-dimension mismatch. Current public SGLang and vLLM default branches do not expose a GLM-5.3 engine/MTP mapping that can replace the purpose-built image, so no speculative tensor remap was guessed. The sealed final_mtp.json records the exact scope, commits, image identity, and failure signature.

Refusal stress changed from 21 broad refusals in 64 sampled responses to 0 broad refusals and 0 anomaly flags over all 1684 final responses (842 harmful + 842 harmless).

DFlash2

incoai/GLM-5.3-Flash-DFlash2@dc77ff1c99eeb2df044ee3d4f0094eb033fee410 was tested as a local pinned speculative companion with block size 8. It is licensed CC BY-NC-ND 4.0 and is not included, modified, or redistributed in this repository. Download it separately and review its license before use.

On the measured two-GB10 TP2 runtime, DFlash2 uses eager execution with decode and prefill CUDA graphs disabled and FlashInfer autotuning disabled. The pinned SGLang image includes an audited GLM-5.3 mHC compatibility patch: auxiliary capture uses hidden_states directly when the communicator deliberately returns residual=None, while non-mHC paths retain hidden_states + residual. The graph/autotune-disabled settings are the independently measured GB10 profile, not a substitute for that correctness fix.

Runtime recipe

The tested image tag is superglm53-sglang:glm53-dflash2-gb10-mhcfix.

  • spark-deca image identity: superglm53-sglang:glm53-dflash2-gb10-mhcfix
  • spark-dd73 image identity: superglm53-sglang:glm53-dflash2-gb10-mhcfix

On each node, keep an identical copy of this repository and the pinned DFlash2 directory. Start rank 1 first, then rank 0:

# spark-dd73
SUPERGLM53_SPECULATION=dflash /home/psw/work/superglm53/bin/serve_superglm53_sglang.sh \
  1 /home/psw/work/superglm53/final/SuperGLM-5.3-Flash-NVFP4-DGX-Spark \
  SuperGLM-5.3-Flash-NVFP4-DGX-Spark 8888 \
  /home/psw/work/superglm53/dflash2_dc77ff1c

# spark-deca
SUPERGLM53_SPECULATION=dflash /home/nvidia/work/superglm53/bin/serve_superglm53_sglang.sh \
  0 /home/nvidia/work/superglm53/final/SuperGLM-5.3-Flash-NVFP4-DGX-Spark \
  SuperGLM-5.3-Flash-NVFP4-DGX-Spark 8888 \
  /home/nvidia/work/superglm53/dflash2_dc77ff1c

Use SUPERGLM53_SPECULATION=none for autoregressive decoding. Native MTP is intentionally not advertised on this two-node GB10 TP2 topology. The pinned launch script uses DSA/TileLang attention, FlashInfer CUTLASS MoE, BF16 KV cache, dynamically discovers the RoCE v2 GID, applies the audited mHC null-residual fix, and disables CUDA graphs and FlashInfer autotuning only for the measured DFlash2 profile.

The sealed profile defaults to one active request. Two-request hybrid KDA/Mamba decoding produced rare non-persistent U+FFFD detokenization anomalies during stress testing; the serialized profile and hash-only sequential anomaly rechecks eliminated them. Raise SUPERGLM53_MAX_RUNNING_REQUESTS only after validating the exact runtime revision and workload.

Compatibility

  • SGLang on two GB10 nodes: supported by the pinned, hash-recorded runtime image and measured here.
  • GGUF/llama.cpp: no release; inspected llama.cpp ca3d5a3e10d53f7ea672cb9b6178faca3e2807bc has no glm5_next converter mapping.
  • MLX: no release; inspected MLX-LM and MLX-VLM revisions expose no glm5_next runtime.
  • vLLM on GB10: not claimed; the inspected path produced incorrect prompt-echo output.

Evidence identities

The repository includes release_evidence/ with the derivation manifest, upstream/architecture pins, baseline and final benchmark summaries, full hash-only refusal stress, and runtime comparison. SHA256SUMS.json seals every published byte. Remote LFS hashes, the exact private clean-reload commit, anonymous public inventory, and immutable v1.0.0 tag are sealed by the publication report.

Limitations and responsible use

This is a structural intervention, not a general fine-tune. It may change refusal behavior and should be evaluated for each deployment. No claim is made beyond the measured hardware, runtime, context length, and behavior gates above. Users remain responsible for applicable laws, upstream terms, model risks, and output verification.

Attribution

  • Official GLM-5.3-Flash: zai-org/GLM-5.3-Flash
  • NVFP4 base conversion: LibertAIDAI/GLM-5.3-Flash-NVFP4
  • NVFP4 donor: dealignai/GLM-5.3-Flash-UNCENSORED-NVFP4
  • Local speculative companion: incoai/GLM-5.3-Flash-DFlash2 (not redistributed)

The checkpoint inherits the upstream MIT license. DFlash2 remains under its separate CC BY-NC-ND 4.0 terms.