AliceAI-Foundation-80B-A3B-Base-GGUF

GGUF port of yandex/AliceAI-Foundation-80B-A3B-Base: MoE, 80B total / 3B active parameters, tokenizer embedded in every file. This is a base model — no chat template, not instruction-tuned; it continues raw text.

Requires a llama.cpp fork. The alice_ai architecture is not in upstream llama.cpp; a stock build fails with unknown model architecture. Use robolamp/llama.cpp at commit 9a9ca8b. This notice will be removed once the port lands upstream.

UPD: now supports MTP, recommended --spec-draft-n-max for Q4 is 2, for Q8 use 3.

Variants

Full file names: AliceAI-Foundation-80B-A3B-Base-<variant>.gguf.

¹ Total variation distance between next-token distributions and an f32 reference on a held-out prompt set; lower is better. Measured on the full model for Q8_0, on a truncated probe for the other rows. The port itself matches the HF f32 reference within 3.48e-05 TVD (100% head).

² Share of mass-carrying tokens (p ≥ 0.01) whose top choice is identical.

³ Number of forced chosen-token ties.

• Not published: Q3_K_M. Head overlap drops to 50% and greedy generation degenerates on some prompts.
• No IQ* / imatrix variants: no importance matrix exists for this model yet.

Conversion notes

• The tokenizer follows transformers, not the SentencePiece proto (they disagree on leading whitespace), via a new optional key tokenizer.ggml.spm_prepend_hf (default off, other models unaffected). 1028 USER_DEFINED vocabulary pieces are promoted, so control tokens like [COT_ENABLE] are not fragmented.

License

Derivative of yandex/AliceAI-Foundation-80B-A3B-Base — the original license applies. Architecture port, conversion and verification: this repository's author, Yulia Yakovleva.