GLM-5.3-Flash Uncensored Mixed Quant GGUF

Mixed-precision GGUF derived from orcarouter/GLM-5.3-Flash-Uncensored-FP8. The recipe preserves higher precision in always-active paths and six boundary routed layers, with IQ2_XXS as the minimum precision.

Status: published. File identity and bounded single-GPU image checks are complete. Bounded Spark runtime checks and matched performance rounds are recorded below, including completed native 1M input and cached Rust HTTP continuations with resident and SSD paths. Broader quality remains open.

Quantization recipe

Layer indices are zero-based: trunk 0–44, MTP 45. Trunk layers 0–2 are dense.

Gate/up use the same format within each layer. Each tensor uses one block format throughout; routed matrix widths remain divisible by 256. Source-weight reconstruction proxies were collected for all 126 trunk expert matrices.

Q2_K does not require an imatrix; optional importance weighting uses available activation statistics without changing the format or file size.

Calibration covered 81,920 text positions; every trunk routed expert was observed. A bounded Q2_K source-weight sample showed 51.2% lower activation-weighted squared reconstruction error (SSE) with imatrix. This is a reconstruction proxy, not a measured response-quality gain. See calibration-summary.json.

Calibration uses GLM-native text inputs and relevant source references from Inkling-Small Multimodal Calibration. The imatrix is collected from text inputs. Original images are prepared for separate image quality checks; image-conditioned activation calibration is not claimed. MTP activation statistics are not collected by the current upstream GLM graph; block 45 uses weight-based importance for IQ2_XS gate/up and standard unweighted Q2_K down. Drafting behavior remains a target-hardware check.

Runtime

This GGUF targets the native GLM format in antirez/ds4. The mixed expert combinations need the accompanying runtime extension; upstream llama.cpp compatibility is not established. Build and usage commands are in runtime/README.md.

The Rust-host runtime ds4-dfm-rs adds this mixed recipe on DGX Spark. SSD expert streaming is optional and defaults to Off. One raw VMM owner and one resident worker pass bounded text, tool, image, state-restore and disk-reuse checks. Embedded MTP is opt-in; accepted-prefix state is checked separately from drafting quality.

Spark performance update (2026-10-08)

The Prefill improvements are included in ds4-dfm-rs PR #79. Fresh serial runs on DGX Spark / GB10 use CUDA 13.3.73, a 32K context allocation, natural Italian text, 64 greedy output tokens and MTP Off. The 512-token boot prewarm is outside timing. The GPU clock range stays 300–2200 MHz; active cold SSD clocks are 2184–2197 MHz. Dense FP16 GEMM stays Off.

Resident physical Prefill batches increase from 128 to 2048 rows. Repeated controlled Resident runs measure 266.43–266.98 tok/s at 2K input, 211.74–211.88 at 8K and 201.37–201.38 at 16K, with Decode 14.68–14.83 tok/s. These repeats and the SSD arms set DS4_GLM53_Q2_WORKLIST=0; the first table row keeps its native default.

Both SSD A/B arms already use 2048 physical rows. The retained arm adds a checkpoint-aligned 4096-token layer-major window and next-layer read/upload overlap. Cold arms start with zero resident file pages verified by mincore, then use advisory expert-page eviction. Cold SSD Decode varies with read latency. The fixed pool uses 3.97 GiB mandatory tensor spans plus about 24 GiB expert slots, compared with 87.46 GiB resident raw VMM weights. Active KV, state, workspace and headroom are additional memory in both paths.

A separate final cold Auto-cache 8K record measures 56.75 Prefill / 6.38 Decode tok/s, funding 72.73 GiB of expert slots. Auto capacity depends on available memory, context, banks and optional state; this single run is not an Auto-versus-fixed speed multiplier.

Same-width supply/preparation A/B passes full-logit and 64-token comparisons; representative answers and restore checks are recorded separately. This raw-text speed fixture is not a broad quality evaluation and does not extend the older 1M qualification to the changed Prefill path. Exact measured source/build hashes, controls and limitations are in the report and per-run evidence.

Earlier Spark rounds (2026-10-07)

Matched fresh-worker rounds improve 8K prefill by 3.08% and 8.91%, 8K decode by 1.77%, and 2K decode by 0.47%. Each round has its own retained baseline; these percentages are not a cumulative claim. SSD and MTP are Off for timing. The retained 8K medians are 137.31 prefill and 14.70 decode tokens/s. All 154880 F32 logits and 64 greedy IDs match across paired arms.

Per-arm clocks, settings and build identities and runtime/gate details record the exact scope. Implementation PR. Final resident API checks pass 24 short requests across Chat, Responses and Messages, including tool continuations and actual red/blue images. Three bounded Spark OCR fixtures also pass. General text/image quality remains unqualified. An earlier forced-keep4 state difference remains unexplained despite passing repeats and separate actual-acceptance gates.

Native Spark execution completes 1,048,512 input tokens in a 1,048,576-token allocation with SSD Off, one raw VMM owner and one worker. All three retrieval codes are correct; full-prefix FILE restore and 32 generated-token replay preserve serialized state exactly. Minimum availability is 13.39 GiB. Repeated ASCII filler is a context diagnostic, not natural-text throughput. Actual On3 accepted-prefix gates pass at native 1M, Rust 6147 and official 544-token inputs, with maximum accepted counts of 4, 4 and 2. Full state, logits and generated IDs match ordinary target transitions; restore is exact. Fresh Rust HTTP gates reuse 1,048,512 tokens for three questions with SSD Off and one resident worker, or SSD On with a 24 GiB cache and two banks. All codes are correct; MTP On3 and ordinary top-k1 agree on answers and completion counts within each profile. A separate short red-image request passes beside the SSD long text banks. Minimum availability is 13.33 GiB resident and 59.61 GiB SSD. These are trusted checkpoint continuations, without a cold HTTP 1M prefill or a 1M image prompt. Restored bank prefixes remain reusable after edited queries. A separate 8K raw-owner/two-bank edit reuses 4096 tokens across banks, preserves the source and matches fresh reuse Off answers/counts. Active KV remains resident; disk checkpoints persist inactive sessions. Image serving requires the BF16 encoder. Native video input is not established.

The source model's uncensored behavior and benchmark results are not new measurements of this quantization. Quality results will identify the exact tested build and settings.

The H100 section below records the original artifact-release checks.

Bounded verification

Both file hashes and all 1,412 main tensor entries were verified after public upload. Re-encoding samples matched exactly across 129 routed tensors and 516 rows. Eight reserved DocVQA/ChartQA images completed on one H100 with SSD expert streaming: 8/8 requested answers matched, with 15/16 manual rubric points. One reply added an incorrect steady-decline claim to a correct peak-year answer. These are bounded checks. General text, MTP and long-context qualification remain pending. Exact settings and runtime identities are in verification-summary.json.

License

MIT, inherited from the source checkpoint. See LICENSE.