Standard quantization of the original checkpoint at fa381f7c466739475e172a7d3234e77351e4f613. The 200 decoder linears are 8.0 bpw; the input embedding, vision and native draft remain 8-bit, and the output head remains BF16. Original RoPE is unchanged.
One 64 GiB GPU ran bounded waves of four and eight requests with 1,048,576 shared Q8 tokens and the original 524,288-token request ceiling. See RELEASE-VERIFICATION.json for measured memory and timings. The checks did not fill the pool with long request histories.
Use the bundled SenseNova-capable ExLlamaV3 source. The measured Meshify profile uses target-only decoding and remains stopped until requested.
model-00001-of-00003.safetensors 4.3 GBmodel-00002-of-00003.safetensors 3.7 GBmodel-00003-of-00003.safetensors 2.7 GB