Flint Heretic — MLX, 4-bit

A copy of Recursive Flint with most of its refusals removed, quantised to 4-bit for Apple Silicon.

This is an abliterated model. It refuses far less than the original. Read Limitations before you use it.

What this is

Flint is a 39.8-billion-parameter mixture-of-experts language model from Recursive. It writes out its reasoning before it answers. It is Apache-2.0 and was released in bf16.

This repository holds that model with its refusal behaviour mostly ablated — the technique Heretic popularised: compute the direction in activation space that the model uses when it declines a request, then remove that direction from the weights.

Quantisation is the only change beyond the ablation. The rounding is MLX's standard affine 4-bit with group size 64.

What was changed

Fifty weight matrices, in layers 24 to 48 of the 49-layer stack — the later two thirds of the network:

• 25 × self_attn.o_proj (attention output)
• 25 × mlp.shared_expert.down_proj (shared expert output)

The 256 routed experts in each layer were not touched — only matrices that write directly into the residual stream. Layers 0 and 1 use a dense mlp.down_proj instead of a shared expert, and fall outside the window in any case.

For each row of each of those matrices, the component along a per-layer refusal direction was subtracted, and the row was then rescaled to its original length. Preserving the row norm is what keeps the model coherent: without the rescale the network produces gibberish.

The refusal directions come from Heretic's method — mean activation over withheld harmful prompts minus mean activation over harmless prompts, per layer. The ablation strength was a flat λ = 4.0 across all 25 layers, with nothing applied below layer 24. Heretic's default is a smooth ramp; a hard window below layer 24 scored better on refusals and KL.

In plain words: we found the direction Flint moves in when it decides to refuse, and we subtracted it from the weights that write into that direction, in the back two thirds of the model.

Results

Held-out prompts only. 100 harmful prompts that were not used to compute the directions, greedy decoding, 100-token budget, measured on this exact 4-bit checkpoint. (The capability benchmark below uses the recommended sampling settings instead.)

Refusals fell 4.3×, and the 20-question maths check is unchanged. The cost is visible in the last row: the ablated model reasons for much longer, and 49 of 100 responses never close the reasoning block inside the answer budget. Almost all the residual refusals live in that failure mode — 18 of the 22 come from the 49 unclosed responses.

Capability benchmark

Base versus Heretic, both 4-bit MLX, same harness, with Recursive's recommended settings (temperature 1.0, top-p 0.95, top-k 20, up to 32,768 output tokens). Fixed subsets, scored by exact answer; 95% Wilson intervals in brackets.

Heretic matches the base on GSM8K and MMLU-Pro and scores a little higher on CulturalRiddles. GPQA Diamond drops from 76% to 62%. The intervals overlap at this sample size, but treat the hardest science questions as the place where the ablation costs capability.

Speed

Measured with the lab's plain MLX implementation on an M5 Pro (64 GB): about 35 tokens/s on one stream and about 150 tokens/s across eight streams. rec, Recursive's engine below, is faster; it has not been measured on this checkpoint yet.

Requirements

• Apple Silicon Mac (M1 or later).
• 48 GB of unified memory minimum, 64 GB or more comfortable for long contexts.
• About 25 GB of free disk for the checkpoint.
• Python 3.12 or newer, and Xcode command line tools.

Install and run

Download the checkpoint:

hf download balllvin/flint-heretic-mlx-4bit --local-dir flint-heretic-int4

Get the engine — rec, Recursive's own MLX runtime. It accepts a local checkpoint through --model-dir plus --manifest:

git clone https://github.com/recursive-oss/rec && cd rec
python3.12 -m venv .venv && .venv/bin/pip install -e engine

Chat with it:

.venv/bin/rec-mlx chat flint \
    --model-dir ../flint-heretic-int4 \
    --manifest ../flint-heretic-int4/flint-int4.json \
    --context-limit 65536

Or serve an OpenAI-compatible API:

.venv/bin/rec-mlx serve flint \
    --model-dir ../flint-heretic-int4 \
    --manifest ../flint-heretic-int4/flint-int4.json \
    --port 8080 --context-limit 65536

rec's default context limit is 8,192 tokens, which is too short for Flint's reasoning; the examples set 65,536, and up to 262,144 works on a 64 GB Mac.

flint-int4.json is the manifest rec validates the files against. It lists the sha256 of every checkpoint file here, regenerated after the ablation, so rec checks these exact bytes and nothing needs to be rebuilt locally. If pip install -e engine fails because it wants to compile the grammar module, pass --grammar none — the model does not need it.

Recommended generation settings

These are the settings Recursive recommends for Flint, and they were used for every number on this page. The model thinks before it answers; a 100-token budget will often cut it off mid-reasoning, so give it room.

Limitations

• This model is deliberately less safe than the original. That is the whole point of the change, and it is your responsibility to put guardrails around it if you deploy it.
• It will comply with requests the base model refuses. Some of those completions will be harmful. There is no moderation layer in front of it.
• Abliteration is a blunt edit. Expect long, sometimes rambling reasoning, occasional repetitions, and weaker instruction-following in the affected layers' domains.
• The capability checks are small samples (40 to 70 questions per benchmark). They are evidence, not a guarantee. GPQA Diamond fell from 76% to 62%.
• 4-bit quantisation adds its own small degradation on top of the ablation.
• No fine-tuning follows the ablation. Nothing was done to repair capability.
• Flint's training data and its limitations carry over unchanged.

Licence

Apache-2.0, inherited from RecursiveSI/flint. The ablation changed weights; it did not change the terms.

Attribution:

• Flint — Recursive, https://huggingface.co/RecursiveSI/flint, Apache-2.0.
• Heretic — the ablation method, Philipp Emanuel Weidmann and contributors, https://github.com/p-e-w/heretic, AGPL-3.0. This model was produced with a from-scratch reimplementation of Heretic's method; no Heretic code is redistributed here.
• rec — the MLX runtime used to serve it, https://github.com/recursive-oss/rec, Apache-2.0.