Flint INT4 GGUF
Q4_1 GGUF weights for Recursive Flint, primarily intended for Apple Silicon using llama.cpp with Metal. The 26.0 GB file includes weights, tokenizer, and chat template.
See Flint’s model card for model details, benchmarks, chat conventions, and limitations.
Run on Apple Silicon
Requires an Apple Silicon Mac with at least 48 GB of unified memory.
Follow llama.cpp’s Metal installation/build guide, using a release with Flint support, and put llama-server on your PATH. Use Metal GPU offload with -ngl all.
With the Hugging Face CLI installed:
hf download RecursiveSI/flint-int4-gguf flint-int4-q4_1.gguf pi-models.json opencode.json \
--local-dir flint-int4-gguf
llama-server -m flint-int4-gguf/flint-int4-q4_1.gguf \
--alias flint -ngl all -c 262144 -np 1 --cache-ram 0 \
-b 256 -ub 128 -t 8 -fa on -ctk f16 -ctv f16 --jinja \
--temp 1 --top-p 0.95 --top-k 20 --min-p 0 \
--host 127.0.0.1 --port 8080
The example uses the full 262,144-token context with F16 KV. Reduce -c only if you need to lower KV memory use. Leave --swa-full unset and retain --cache-ram 0 to limit extra host-memory use. No sliding-window or RoPE override is needed.
Use with pi or OpenCode
Keep llama-server running and open another terminal in the directory containing the downloaded model folder.
pi
Install pi using its installation guide, then run:
mkdir -p .pi-flint
cp flint-int4-gguf/pi-models.json .pi-flint/models.json
PI_CODING_AGENT_DIR="$PWD/.pi-flint" pi --provider flint-local --model flint
See pi’s custom-model guide for configuration options.
OpenCode
Install OpenCode using its installation guide, then run:
OPENCODE_CONFIG="$PWD/flint-int4-gguf/opencode.json" opencode --standalone
See OpenCode’s provider guide for configuration options.
The client configs use 262,144 context tokens and a configurable 65,536-token generation budget, including reasoning. Prompt and output share the total context. If you change -c, match the client context limit to it.
Format notes
Q4_1 stores the MLX export’s trained INT4 codes with FP16 scales and offsets. Embeddings and other floating tensors are stored as F32, making this file larger than Flint INT4.
Use --jinja and preserve reasoning_content when replaying conversation history.
Training data
Flint’s training data included publicly available and crawled web documents, text and code, commercially licensed data, and AI-generated synthetic data. Post-training included instruction tuning, reinforcement learning and human-preference data. See the base Flint model card for shared model and training information.