Runtime: Osaurus 0.25.21 or newer (osaurus.json: required_osaurus_version; model_version 3 = these JANGHT weights). This repository was OsaurusAI/GLM-5.3-Flash-JANGH2 until 2026-10-13; old links redirect here. If you downloaded the JANGH2 weights earlier, this is a different bundle: 4.3 GiB smaller, closer to the original model, with a speculative-decoding drafter included.

OsaurusAI/GLM-5.3-Flash-JANGHT2.4

GLM-5.3-Flash for 128 GB Macs, now 4.3 GiB smaller and closer to the original model than our JANGH2 release, with a speculative-decoding drafter in the box.

A JANGHT bundle of zai-org/GLM-5.3-Flash: a 300B-class MoE (288 routed experts, top-8 + shared expert) with KDA linear attention, sparse attention, and vision + video towers.

• Routed experts: JANGHT, a per-layer mix of two formats, chosen by measurement:
  • JANGT: a trellis code at 2, 2.5 or 3 bits per weight (Hadamard-128 rotation, 12-bit state). The 2.5-bit variant alternates 6- and 4-bit transitions, so a layer can sit between 2 and 3 bits.
  • JANGH: our Hadamard-32 codebook format at 2, 3 or 4 bits, rounded with GPTQ.
• Everything else: 8-bit (MXFP8 where the source weights sit on an FP8-MX grid, affine 8-bit elsewhere) or full precision. Vision tower: bf16.
• DFlash2 drafter (dflash2/): z-lab's block-diffusion draft model for GLM-5.3-Flash, unmodified. Osaurus uses it for lossless speculative decoding; code and other low-entropy text get much faster.
• Name: JANGHT + 2.4 bits per weight averaged over the whole model (91.57 GiB × 8 / 321.3B source parameters), drafter excluded.

Quality vs the bf16 model

123 held-out sequences, 95,025 teacher-forced positions, scored against the full bf16 model run layer by layer from disk. Top-128 renormalized KL. Every row uses the same 8-bit non-expert weights, so the rows differ only in the routed experts. The set is hard on purpose: 44k of the positions are multi-turn tool conversations, plus long context and Chinese.

Earlier comparison: official FP8 reference, 20 short prompts (from the JANGH2 release)

15,830 teacher-forced positions, top-128 renormalized KL against the official FP8 release. This is a different, much easier set than the table above, and JANGHT2.4 was not run on it; the rows are kept for the comparison with other public quants.

¹ Same 20 prompts (15,850 positions), loading its shipped quantized weights natively.

On the 95k-position set above, JANGHT2.4 beats JANGH2 on every aggregate (median KL −15%, top-1 +2.1 points).

Earlier fix kept in JANGHT2.4: long reasoning ends on its own (JANGH2 revision 2)

JANGH2 revision 1 could not end long reasoning: error-minimizing row scales shrank every low-bit matrix (12% at 2 bits), which suppressed </think> after a few thousand reasoning tokens. Revision 2 corrected the row scales to unit gain, and JANGHT2.4 applies the same rule to every expert unit.

By domain, median KL and top-1 agreement:

Tool decisions

Inside the 95k positions: 109 points where the bf16 model's transcript calls a tool, 96 where it answers.

Reasoning ends on its own

16 held-out transcripts written by the bf16 model in its native format (long design and coding reasoning, multi-step tool conversations), 60,243 assistant positions, scored against bf16:

Free generation on a long design prompt (served, effort low, three seeds): the model closed its reasoning on its own in 3 / 3 runs, after 8.5k, 9.7k and 12.4k reasoning tokens.

Live behavior (served, temperature 0, with and without the drafter)

These suites check behavior; every bundle that works passes them.

Speed (M5 Max 128 GB, reference measurements)

Measured with the development server the bundle was validated on, one request at a time; two interleaved runs per bundle (A, B, A, B), a fresh server for each run after a 3-minute cool-down, each run the median of 3 probes on a never-seen prompt. Osaurus figures on your Mac will differ in absolute terms; the comparison between the two bundles is the point.

Decode is on par with JANGH2 in a 4.3 GiB smaller bundle. Long-prompt prefill is ~8% slower (the trellis tiles cost more to decode than the JANGH codebook).

With the bundled DFlash2 drafter decode depends on how predictable the text is: 43.5 tok/s on prose, 65-68 tok/s on code, same conditions. Output is unchanged.

What's in the bundle

• Experts (42 MoE layers):

  projection	JANGT 2	JANGT 2.5	JANGT 3	JANGH 2	JANGH 3	JANGH 4
  gate / up	35 layers	4	2	1	—	—
  down	1	19	—	10	8	4

• Vision + video: the bf16 vision tower (byte-identical to JANGH2) + the consolidated image/video processor config.

• dflash2/: the DFlash2 drafter (2.18 GiB), unmodified, under its own license (CC BY-NC-ND 4.0, see dflash2/README.md). Osaurus loads it with the model and adapts the draft length to how often drafts are accepted; output is identical to plain decoding at temperature 0 and has the same distribution when sampling.

• No MTP: layer 45 is omitted; its bytes went into expert precision.

• Thinking + agentic:

  • Thinking is ON by default (the template opens <think>).
  • Reasoning efforts are low / high / max (default max). There is no medium, and no thinking-off mode: the template renders any other value, and a missing value, as Max.
  • Budget for thinking: 8-12k tokens at low on a large design task, 17-23k at max. Use low or high for interactive work and a generous max_tokens.
  • clear_thinking=false preserves thinking in history.

• Tool calls: GLM's XML dialect (<tool_call>name<arg_key>…</arg_key><arg_value>…</arg_value></tool_call>), declared as tool_parser: glm_xml_args; tool results render as <|observation|>. Hermes-style JSON parsers will not work.

• Self-describing:

  • config.json carries the per-module quantization map: 102 JANGT expert projections ("mode": "jangt", "kbits"), 24 JANGH ("mode": "jangtq2", "bits"), 147 MXFP8, 214 affine 8-bit. The jangt block holds the trellis parameters and the per-layer outlier columns; the jangtq block (version 2) holds the JANGH codebooks.
  • jang_config.json records calibration, the allocation plan, the drafter and capabilities.

• Memory: 91.57 GiB of weights + 2.18 GiB drafter. Fixed-size linear-attention state + compressed-latent KV (~6 KB/token), so long contexts do not balloon memory.

• Every shard is alignment-safe (zero-copy memory mapping). 18 shards.

Serving contract

• Sampling: temperature=1.0, top_p=0.95 (vendor defaults), no repetition penalty
• EOS: [154820, 154827, 154829] · context: 1M native
• Reasoning: reasoning_effort chat-template kwarg (low / high / max), default max
• JANGH units keep their original on-disk identifiers (jangtq block, "mode": "jangtq2", *.tq2_packed / *.tq2_scales). They are not loadable by, and must not be routed to, JANGTQ v1 loaders.

Build details

• Source: zai-org/GLM-5.3-Flash-BF16 @ a5b45eb
• Calibration: 552k tokens rendered with the GLM chat template (coding, tool conversations, cybersecurity, agentic, general, Chinese, science, academic, long context, systems), referenced to the bf16 model run layer by layer; every tenth calibration sequence and all evaluation sets held out
• Allocation: per projection and layer, the format and width with the lowest measured cost on held-out rows, each layer's error weighted by its measured effect on the output distribution
• Row scales: unit gain along the source row on every expert unit (the JANGH2 revision-2 rule)
• Drafter: z-lab GLM-5.3-Flash DFlash2, included unmodified

Quantized and validated by Jinho Jang — eric@osaurus.ai