Sakura Qwen3.8 Flash-Next Compact · ISTA Q2_0 · 512E · CP2

47.56 GiB main model · 512 experts · Windows / Vulkan · MTP included

A smaller experimental community edition of ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF (Q2_0 quant). The two main-model files shrink from 61.86 to 47.56 GiB, saving 14.30 GiB (23.1%), while retaining all 512 routed experts and 16 lookup heads of the unpruned base model. The matching custom runtime and materialized 512E MTP adapter are included separately.

This is the unpruned Qwen base model (NOT Swift, NOT REAP).

Use the supplied runtime and START.cmd. This experimental custom format requires its matching loader; generic GGUF launch instructions shown automatically by Hugging Face do not establish compatibility.

Made and tested on AMD Strix Halo, Radeon 8060S, 64 GB UMA, Windows x64, using Vulkan. Independent community release; not an official Qwen or ISTA-DASLab release.

At a glance

Disk size is not running memory. The tested 8K profile with MTP2 and Q8_0 KV is dimensioned for unified memory systems such as Strix Halo 64 GB UMA. A 47.6 GiB file size does not imply that this model fits a 32 GB or 24 GB GPU.

Hardware compatibility

The status below applies to this Compact CP2 v1 model and the supplied binaries. Vulkan support in a GPU driver alone does not establish model or runtime compatibility.

CPU, memory and storage

• CPU: the runtime was compiled as a native build on the test machine and tested on its AVX-512-capable x64 CPU. A portable AVX2-only build is not included. An AVX-512 label by itself is not a compatibility test for every other CPU.
• Unified memory: 64 GB UMA is the tested configuration, not a measured minimum for every platform. Dedicated and shared Windows counters both draw from that same physical memory pool on Strix Halo. Leave room for Windows, CPU allocations and other applications.
• Discrete GPUs: separate system RAM does not provide the same behavior or bandwidth as Strix Halo UMA. Neither 48 GB nor a larger VRAM capacity alone guarantees compatibility; the device, driver, CPU and custom runtime all need validation.
• Storage: allow at least the 50.00 GiB model-plus-MTP footprint, plus the runtime and supporting files. An SSD is recommended for the lazily read table. Free disk space does not substitute for sufficient runtime memory.

Provenance and what changed

The model tree points directly to the common Qwen base. Its actual lineage is:

1. Qwen/Qwen3.8-Flash-Next.
2. ISTA-DASLab/Qwen3.8-Flash-Next-GSQ-RCO-GGUF (Q2_0 quantization).
3. This Compact CP2 v1 conversion.

Important distinctions:

• No Swift contribution: Unlike the sister release webmp3/Sakura-Qwen3.8-Flash-Next-Compact-Swift-IQ3_XXS-352E-CP2-GGUF, this release is based directly on the unpruned Qwen base quant.
• 512 experts: All 512 routed experts are preserved. No expert pruning (REAP/Swift) was applied.
• Tensor preservation: All expert weights and all non-lookup tensor bytes are 100% byte-identical to the source ISTA Q2_0 model. Only the large per-layer token lookup table was compressed using the CP2 lookup format, and associated GGUF metadata/filenames were updated.
• Tokenizer preservation: Target model's original tokenizer and chat template (<|im_start|>, Jinja template, etc.) are strictly preserved.
• MTP adapter: Materialized for this exact 512-expert model from the shared weights without retraining.

The package also includes a matching native runtime and a materialized MTP adapter for this exact 512E model. See NOTICE.md for the change notice and attribution.

Files

Keep both main shards in the same folder. Select shard 1; the runtime locates shard 2 automatically.

Hugging Face hardware widget: Q2_0 in the filenames identifies the retained source expert-quantization tier; CP2 identifies the custom lookup-table conversion. The widget may group this hybrid release under Q2_0. That label is not a certification of hardware fit or stock-loader compatibility: the included CP2 runtime is still required.

Start on Windows

Download the package with its folder layout intact. Keep the model shards, launchers, mtp/, runtime/ and license files together. No conversion is needed to use the supplied package.

1. Double-click START.cmd in the release folder.
2. Leave its terminal open while using the model.
3. Connect your OpenAI-compatible client to http://127.0.0.1:8080/v1. Query /v1/models for the model ID to use in the client.
4. Stop the server with Ctrl+C in its terminal.

The starter serves a local API only (--no-webui); it does not open a browser chat interface. It binds to 127.0.0.1.

Included profile

Manual equivalent

Run from the release folder in Windows Command Prompt:

set "SAKURA_CP2_GPU_CACHE_MIB=0"
runtime\llama-server.exe ^
  -m Sakura-Qwen3.8-Flash-Next-K512-Q2_0-CP2-v1-Experimental-00001-of-00002.gguf ^
  --host 127.0.0.1 --port 8080 --parallel 1 ^
  --device Vulkan0 -ngl 99 -c 8192 -b 512 -ub 512 ^
  -fa on -ctk q8_0 -ctv q8_0 -lm dio --lazy-mode on --jinja ^
  --spec-type draft-mtp ^
  --spec-draft-model mtp\mtp-Qwen3.8-Flash-Next-K512-materialized-Agention.gguf ^
  --spec-draft-ngl 99 --spec-draft-device Vulkan0 ^
  --spec-draft-n-max 2 --spec-draft-n-min 0 --no-webui

Use the provided 512E adapter with this model. Other shared MTP heads are not interchangeable.

Measured quality: original ISTA Q2_0 versus Compact CP2

Own measurements on these exact conversion inputs and outputs. Lower perplexity is better.

Each slice used 512-token chunks. Corpus identifiers are recorded in results.json. In English and Code, perplexity remained virtually flat (-0.30% / -0.26%). In German, perplexity rose by +1.02% on 8 chunks and +1.19% on 40 chunks; these deviations are retained and openly documented. Per user priority, testing focused on English and code without further German optimization. These small changes do not establish general quality superiority.

Generation speed and MTP speculation: own tests

Hardware: AMD Radeon 8060S, Strix Halo, 64 GB UMA, Windows, Vulkan. Tests dated 20 October 2026.

• Generation speed: approximately 33–37 token/s under sustained generation with MTP2; short 256-token smoke tests reached 38.92 / 43.78 token/s (English) and 41.79 token/s (Code).
• MTP speculation acceptance: 64.8–69.5% with the supplied materialized 512E MTP adapter (--spec-type draft-mtp --spec-draft-n-max 2).
• Answer parity: Greedy short answer checks (e.g. Paris, 42) matched identically with and without MTP.

A broad standardized coding correctness evaluation has not been performed for this conversion. The model is an experimental release; no quality or correctness guarantee is claimed.

Compatibility and known limits

• Matching custom runtime required. CP2 v1 is an experimental custom tensor format, not a stock GGUF quantization preset. Stock llama.cpp, Maya, LM Studio, Ollama and unrelated forks have not been qualified for this package.
• The included native binaries were tested on Windows x64, an AVX-512-capable CPU and Radeon 8060S with Vulkan. Other CPUs, GPUs, Linux and macOS are untested. Do not mix DLLs from unrelated runtime packages.
• The runtime derives from Agention/llama.cpp and ggml-org/llama.cpp. The supplied build adds support for this Compact model. It has not been demonstrated to be the fastest available fork.
• Deep-context correctness above 8K, 128K operation, multiple concurrent slots, tool use and vision are unqualified for this conversion. No vision projector is included.
• Compact retains all 512 experts of the base model and adds lossy lookup-table compression.
• The supplied starter is a tested starting profile on one machine. Model file size alone is not a system-memory or VRAM requirement.

Licenses and credits

Model weights retain the Qwen Community License 1.0. Read the original license text before deployment. The native software's MIT and third-party notices are included under runtime/; software licenses do not replace the model-weight licenses.

Thanks to Qwen, ISTA-DASLab, Agention and the llama.cpp / GGML contributors for the upstream model, quantization and runtime work. Keep the supplied license texts and NOTICE.md when redistributing.

---

中文简要说明 · 樱花 Compact 512E

这是基于 ISTA-DASLab GSQ-RCO Q2_0 的全 512 专家版 Sakura 实验性紧凑版本。主模型的两个 GGUF 分片从 61.86 GiB 缩小到 47.56 GiB，节省 14.30 GiB（23.1%）。保留未剪枝基础模型的全部 512 个专家和 16 个查找头；专家权重及其他非查找表张量字节 100% 相同。非 Swift 模型，无 REAP 剪枝。 查找表采用有损 CP2 压缩。

请使用本仓库附带的运行时和 START.cmd。 512E 专属 MTP 适配器另占 2.44 GiB。保持目录结构完整，启动后客户端连接 http://127.0.0.1:8080/v1；启动脚本只提供本地 API，没有网页聊天界面。

测试平台：Windows x64、支持 AVX-512 的 CPU、AMD Strix Halo Radeon 8060S、64 GB 统一内存、Vulkan。测试配置为 8K 上下文容量、单槽位、Q8_0 KV、MTP2，GPU 行缓存关闭。实测持续生成速度约 33–37 token/s，短测试中达 38.9–43.8 token/s。配套 512E MTP 适配器的投机接受率为 64.8–69.5%。贪婪简短答案在启用/禁用 MTP 下完全一致。磁盘大小不等于运行内存需求；其他平台未验证。

英语和代码困惑度基本持平（-0.30% / -0.26%），德语困惑度略有上升（8块 +1.02%，40块 +1.19%）。语料哈希见 results.json。模型处于实验状态，不提供通用质量保证。

完整来源、文件列表、测试条件和限制见上方英文说明。模型权重受附带的 Qwen 许可约束；运行时软件许可不能替代模型许可。这是独立社区版本，不是 Qwen 或 ISTA-DASLab 官方发布。