Gemma 3 1B IT — native Q4_1 for XDNA2
An experimental, custom-quantized derivative of Google's gemma-3-1b-it for
the frozen full26-c32768-selected XDNA2 PDI profile. All 26 transformer
layers and the full 262144-token vocabulary head execute on the NPU. Model
quality is REPORT_ONLY.
Ownership and maintenance
This model bundle and the companion gemma-xdna2 runtime are developed,
maintained, and released by the ETRI On-device AI Compiler (ETRI-OAC)
team. The yongseungyu Hugging Face account is the publication account; it
does not denote individual ownership of the artifacts or the project.
Release contract
| Component | Pinned release |
|---|---|
| Runtime | gemma-xdna2==0.1.0a2 |
| Model repository | yongseungyu/Gemma-3-1B-IT-XDNA2-Q4_1 |
| Model revision | 3098bf6f10549f058ee27e8f7ac14fddd6a97964 |
| Kernel profile | gemma3-1b-q4_1-bf16-i8g32-c32768-selected-v1 |
| Format | gemma-xdna2.native-q4_1.v1 |
Use the listed runtime and exact model revision together. The runtime independently checks the model and packaged-kernel contract; no CPU model fallback is provided.
Requirements
- CPython 3.12 on Linux x86_64.
- glibc >= 2.39 and libstdc++ with
GLIBCXX_3.4.32andCXXABI_1.3.15. - A compatible XDNA2 NPU/driver/firmware combination and read/write access to
its device node (normally
/dev/accel/accel0). - At least 2 GiB of memlock allowance. 4 GiB is the recommended setting for end-to-end validation and is used in the steps below.
The wheel carries a manylinux_2_39_x86_64 tag. That tag describes the
user-space ABI only; it does not qualify a different NPU, driver, or firmware.
Run doctor --probe and the end-to-end commands below on every new host.
Reproducible installation and validation
The following commands install only the published wheel, use the fixed model commit, verify the downloaded bundle, and run NPU inference. The model download is approximately 1.34 GB.
mkdir -p ~/gemma-xdna2-test
cd ~/gemma-xdna2-test
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install \
--index-url https://pypi.org/simple \
--only-binary=gemma-xdna2 \
--no-cache-dir \
"gemma-xdna2==0.1.0a2"
gemma-xdna2 --version
# Omit only when the current terminal already has at least 4 GiB memlock.
sudo prlimit --pid "$$" --memlock=4294967296:4294967296
gemma-xdna2 doctor --probe
gemma-xdna2 download \
--repo yongseungyu/Gemma-3-1B-IT-XDNA2-Q4_1 \
--revision 3098bf6f10549f058ee27e8f7ac14fddd6a97964 \
--output ./gemma-model
gemma-xdna2 verify ./gemma-model
gemma-xdna2 generate \
--model ./gemma-model \
--prompt "What is the capital of France?" \
--max-tokens 32 \
--json
doctor --probe confirms device access and reports the current memlock limit.
verify checks the local model payload and its kernel compatibility before
inference. generate is the actual PDI execution check and writes the answer,
token IDs, and speed information as JSON. The prlimit command affects only
the active terminal and its later child processes; use a PAM limits.d entry
when a persistent policy is required.
Public-release validation
Public-install E2E validation completed on aisbc450 on 2026-11-14.
On ai356 on 2026-11-16, a clean CPython 3.12.3 environment on an XDNA2
host (Linux 7.0.0-38, firmware 1.1.2.64) installed the public PyPI wheel above,
probed /dev/accel/accel0, downloaded the fixed commit, and completed both
verify and generate. The ai356 host had a 3.81 GiB memlock allowance,
above the 2 GiB minimum but below the 4 GiB recommended validation setting.
The France prompt returned The capital of France is Paris! and measured
48.69 decode tokens/s for the 11 measured decode submissions. These are
end-to-end compatibility results for those hosts, not qualification of other
driver or firmware combinations.
Provenance and artifact format
| Item | Value |
|---|---|
| Original model | google/gemma-3-1b-it |
| Original revision | dcc83ea841ab6100d6b47a070329e1ba4cf78752 |
| Original safetensors SHA-256 | 3d4ef8d71c14db7e448a09ebe891cfb6bf32c57a9b44499ae0d1c098e48516b6 |
| Transformer depth | 26 distinct layers |
| Hidden / FFN | 1152 / 6912 |
| Q / KV heads, head dimension | 4 / 1, 256 |
| Context allocation | 32768 tokens; local window 512, global layer every sixth layer |
| Weight format | Native affine Q4_1, groups of 32, BF16 scale and minimum |
| Activation / KV format | Custom int8-g32 packets |
buffers/weights.bin (627609600 bytes) and buffers/constants.bin
(671359488 bytes) are consumed through their manifest ABI. The repository
contains the custom quantized weights, BF16 support tables, tokenizer,
precision metadata, licenses, and validation evidence. PDI/ELF/xclbin and
their parameter set are supplied by the companion binary wheel. The format is
not GGUF and cannot be loaded through AutoModel.from_pretrained or a native
llama.cpp backend. Transformers is used only for tokenization and chat
formatting; PyTorch and MLIR-AIE are not runtime dependencies.
Execution and validation scope
The NPU performs embedding, RMSNorm, RoPE, QKV/projections, attention, GELU/MLP, residuals, KV updates, and the full head. The host writes parameters, starts and waits for the PDI, checks logits for finiteness, and chooses greedy argmax. There is one persistent hardware context and one initial weight load.
Recovery validation covered one-, sixteen-, and twenty-six-layer first-token gates; 27 sequential positions with complete logits and KV cache; and context execution from 128 through 32768 active tokens. For the current packed arithmetic, the recorded first-token and sequential-logit checks match their CPU oracle. Strict current-KV byte equality has known one-code exceptions at 256 and 32768 tokens; they remain documented rather than being hidden or retuned away. The custom packed arithmetic has not been claimed equivalent to unquantized BF16 inference.
Prompt ingestion currently uses sequential full-model NPU steps. Optimized batched prefill, sampling, native llama.cpp integration, and broad quality/perplexity evaluation are future work. Generation is deterministic greedy decoding.
Intended use and terms
This release is intended for research and reproducible experimentation with this specific Gemma/XDNA2 arithmetic contract and frozen PDI. It is not a production-readiness claim or an endorsement by Google or AMD. Users must evaluate generated output for their own application.
Gemma-derived weights remain subject to the Gemma Terms of Use, including the incorporated prohibited-use policy. The bundle includes the required terms, use policy, attribution, and modification notices. Runtime software and its bundled third-party components retain their respective Apache/BSD/MIT/Boost notices.