Kolibri-1 GGUF (Q8 / Q4 / Q2) — DwarfStar format

Quantized GGUF builds of Aleph-Alpha/Kolibri-1 — Aleph Alpha's 78.10B-parameter (3.46B active) German/English MoE reasoning model — converted from the FP8 release at revision e52eb4627d11516b0c01de49210ab5a4e4061444 with DwarfStar's kolibri1_quantize.py. The weights are modified (quantized) relative to the original release.

[!IMPORTANT] These are DwarfStar (ds4) format GGUFs. They load with the ds4 inference engine (Metal or CPU reference) and are not generic llama.cpp-compatible GGUF files.

Suggested order of preference: Q8 (closest to the release, ~78 GB of weights), Q4 (good quality/size trade-off), Q2 (smallest; quantized with the weight-energy bootstrap since no imatrix was used).

Running

Kolibri-1 support is not merged upstream yet; it lives in the main branch of aparusel/ds4-kolibri-1 for now. Build that fork and point it at a file:

git clone https://github.com/aparusel/ds4-kolibri-1 ds4
cd ds4 && make

./ds4 -m /path/to/Kolibri-1-Q4.gguf
./ds4-server -m /path/to/Kolibri-1-Q4.gguf --ctx 32768

Kolibri-1 runs on Metal and the CPU reference backend (ds4 --cpu -m ...). CUDA, ROCm, tensor/pipeline parallelism, SSD streaming, speculation and vision are not supported for this model. Thinking is on by default; --think-level N maps to the released low/medium/high effort wording, and the model defaults to temperature 1, top-p 0.97, top-k 128. See docs/MODELS.md for details.

How these were produced

make -C gguf-tools libds4quants.dylib
python3 gguf-tools/kolibri1_quantize.py \
  --hf models/Kolibri-1 \
  --source-revision e52eb4627d11516b0c01de49210ab5a4e4061444 \
  --out gguf/Kolibri-1-Q4.gguf --quant q4

Every released tensor must be claimed exactly once, and only the exact source snapshot matching the plan is accepted, which is why the source revision is pinned above. Each artifact can be audited with gguf-tools/kolibri1_validate_gguf.py --payload.

License and attribution

The model weights are published by Aleph Alpha GmbH under the Apache 2.0 license; the license text is included in LICENSE. This repository redistributes modified copies (quantized conversions) under the same license, with attribution, and states the changes as required by section 4(b) of the license. It is an unofficial community conversion and is not affiliated with or endorsed by Aleph Alpha.

The original model card contains the Responsible Use section, intended use, risks and limitations; please read it before deployment. The Apache 2.0 grant applies only to the weights and configuration files published by Aleph Alpha — it does not extend to their underlying code, model architecture, parameter settings or training methods.

Conversion tooling is part of DwarfStar (MIT license, © The ds4.c authors and the GGML authors).