Bonsai-27B LiteRT-LM — source repair and experimental conversion
23 September 2026: source reconstruction is repaired and tested; a repaired full-model Android artifact is not yet available. The existing
Bonsai-slice-4L-*files contain only four layers, not the complete 64-layer Bonsai-27B model. The wordfixedin their historical filenames is not an end-to-end quality certification.
Current validation status
| Check | Result | Scope |
|---|---|---|
| Original Q1 source SHA-256 | Pass | Pinned 3,803,452,480-byte GGUF |
| Full source reconstruction | Pass | All 64 layers, all 851 tensors, no missing weights |
| Value-level regression tests | Pass | 21 tests, including exhaustive bit/mask checks |
| Full source generation smoke tests | Recorded in validation report | Deterministic host PyTorch generation, not a benchmark |
| Packed-Q1 numerical prototype | Pass | Small host LiteRT CPU test only |
| Hardware-accelerated Android setup | Pass | x86_64, 8,192 MiB RAM, 30G data partition, WHPX |
| Official Edge Gallery 1.0.19 | Install and launch pass | No repaired-model inference performed |
Repaired full .litertlm export |
Not completed | Export adapter positional-encoding incompatibility |
| Full-model generation in Gallery | Not run | End-to-end release validation does not pass |
The validated source implementation and reproduction commands are in quality/README.md. Machine-readable evidence is in validation/2026-09-23/.
What was wrong
The previous GGUF reconstruction could produce correctly shaped matrices with scrambled values: reshaping flat GGUF storage into its fast-dimension-first dimensions and then transposing is not the correct inverse layout operation. The repair interprets the flat values directly in Hugging Face row-major shape.
The conversion also needs to reverse Qwen's zero-centered RMSNorm transform, restore A_log from the stored negative exponential, and undo the linear-attention value-head permutation. The gated linear-attention norm must not receive the zero-centered norm subtraction. These transforms now have value-level regression tests.
The Q1 sign encoding is byte-major and little-endian, with one FP16 scale for each 128 weights. That bit order was not the cause of the reconstruction failure.
Full-model source repair
The quality/ loader retains the original packed sign bits and scales rather than expanding all 26.90 billion parameters to FP32 or requantizing them to INT8. Embeddings gather only requested rows before decoding; linear layers decode bounded row chunks. Unknown, duplicate, missing or shape-mismatched tensors fail explicitly.
The initial full-source smoke run correctly answered four deterministic prompts, including arithmetic, a capital-city question and exact output. Its parameter/buffer storage was 4,216,748,288 bytes and peak CUDA allocation was 4,642,257,408 bytes on an RTX 4090. These measurements do not establish Android memory fit. The published source validator adds twelve named cases and records actual answers and token IDs; consult its report for the observed result.
The separate small LiteRT prototype passed six numerical comparisons with maximum absolute error 1.049041748046875e-05. A successful tiny graph is not evidence that the complete model exports or runs in Gallery.
Historical artifacts — unchanged
| Files | Contents | Status |
|---|---|---|
experimental_artifacts/Qwen3.5-0.8B-hybrid-exact-c2048.litertlm and c32k/c64k variants |
Qwen3.5-0.8B hybrid graph experiments | Not Bonsai-27B |
experimental_artifacts/Bonsai-slice-4L-Q1dequant-fixed-c2048.litertlm and c32k/c64k variants |
Four-layer Bonsai slices | Legacy graph experiments; not full-model quality releases |
Other Bonsai-slice-4L-* files |
Earlier four-layer experiments | Historical, not recommended for evaluating Bonsai-27B chat quality |
No existing weight artifact is replaced by this source-repair commit. The old experimental/ scripts are preserved for provenance, but their transpose Linears reconstruction is superseded by the strict loader in quality/.
Required before an Android release
The repaired source must pass a compatible full-model export, complete-graph LiteRT parity, tokenizer/stop-token checks, measured memory on the 8 GB x86_64 AVD, and actual generation in official Edge Gallery using the exact exported artifact hash. App installation and launch alone cannot satisfy these gates.