Flint Preview — IQ2XXS/Q2K/Q8 GGUF
This repository contains flint-IQ2XXS-Q2K-Q8.gguf, a community quantization of RecursiveSI/flint. It is not an official Recursive release.
Download and run
Use a recent llama.cpp build with Flint support:
llama-cli -m flint-IQ2XXS-Q2K-Q8.gguf --temp 1.0 --top-p 0.95 --top-k 20 -p "Who are you?"
Quantization recipe
| Tensor class | Quantization |
|---|---|
| Gate and up experts | IQ2_XXS |
| Down experts | Q2_K |
| Other weights | Q8_0 |
The source model is a 39.8B-parameter mixture-of-experts model with about 2.5B active parameters per token. See the source model card for architecture, recommended generation settings, evaluation, and license details.
License
Flint is released under the Apache 2.0 license. See RecursiveSI/flint for the source license and model terms.