flint-opus-step40-GGUF
GGUF of AlexWortega/flint-opus-step40, converted with llama.cpp convert_hf_to_gguf.py
(Flint support, master 4244ad7) and quantized with llama-quantize: BF16, Q8_0, Q4_K_M.
Needs a llama.cpp build with Flint support (PR #30928). The chat prompt format is
<|user|>...<|assistant|><think></think> (empty think = no reasoning, as in training).
Step-40 checkpoint of AlexWortega/flint-opus-curriculum-20261026: a fresh attention-only LoRA (r16/alpha32, q/k/v/o)
on RecursiveSI/flint @ 2a247be, trained with group-relative RL on
antislop tweet continuations graded by an Opus 5.5 judge (astra4 calibrated-v1 reward). Step 40 was the best held-out
antislop point of a 200-step run; later steps regressed.
| held-out (Opus 5.5 judge, 24 prompts each) | base flint | step 40 |
|---|---|---|
| antislop q (0-10) | 4.92 | 5.90 |
| antislop calibrated reward | 0.12 | 0.31 |
| roleplay q (0-10) | 6.38 | 6.13 |
| roleplay calibrated reward | 0.63 | 0.51 |
Small eval sets (n=24) and a single judge: treat these as noisy. Roleplay was not trained before step 101 and is
slightly lower at step 40. The LoRA is folded into the attention weights as W + (B @ A) * alpha / r
(fp32, cast back to bf16), identical to the weights the policy server used during training. Experts are untouched.
Quantization (v2, 2026-10-27)
The first Q8_0 / Q4_K_M uploads used llama.cpp's default rules and drifted noticeably from BF16. They are replaced. KL divergence to this repo's BF16 GGUF (llama-perplexity, ~7k tokens of model outputs):
| file | recipe | mean KLD (v1 → v2) | same top token |
|---|---|---|---|
| Q8_0 | experts Q8_0; embeddings, output, attention, shared experts and dense layers 0-1 kept BF16 | 0.0119 → 0.0077 | 96% |
| Q4_K_M | bartowski's per-tensor layout for flint + bartowski's imatrix | 0.099 → 0.050 | 90% |
Flint is numerically sensitive in llama.cpp: the same BF16 weights on CPU vs GPU already differ by KLD 0.0034.