This repository contains Qwen3-0.6B model, optimized for Renesas X5H platform for text-generation inference.
| Parameter | Qwen3-0.6B |
|---|---|
| NUM_LAYERS | 28 |
| HIDDEN_SIZE | 1024 |
| FFN_DIM | 3072 |
| NUM_HEADS | 16 |
| NUM_KV_HEADS | 8 |
| HEAD_DIM | 128 |
| GROUP_SIZE | 2 |
| VOCAB_SIZE | 151936 |
| RMS_NORM_EPS | 1e-6 |
| ROPE_THETA | 1000000.0 |
| TIE_WORD_EMBEDDINGS | true |
| MAX_POSITION_EMBEDDINGS | 40960 |
The following performance metrics were measured with a prompt.
| Model | Precision | Device | Response Rate (tokens/sec) |
|---|---|---|---|
| Qwen3-0.6B | W4A16 | NPX6 | TBD |
To run model, you need:
Download the installer in any linux PC from Files and versions tab "qwen3-0p6b-w4a16-batched-runner-0.1.0-Linux.sh"
bash ./qwen3-0p6b-w4a16-batched-runner-0.1.0-Linux.sh --prefix=./ --exclude-subdir --skip-license
qwen3-0p6b-w4a16-batched-runner
├── Qwen3-0.6B-f16.gguf
├── firmwares
├── kernel_modules
├── qwen3-0p6b-w4a16
├── qwen3-0p6b-w4a16-batched-runner
├── scripts
└── setup_npu.sh
bash ./setup_npu.sh
./qwen3-0p6b-w4a16-batched-runner -p <PROMPT>
Qwen/Qwen3-0.6B