AJ-Gonzalez/coder-lfm3-1.7b-k3.1-pro
with its refusal behaviour removed by Heretic 2.0.0
(Arbitrary-Rank Ablation). Same architecture, tokenizer and chat template; only the weights differ.
Capability warning. Abliteration costs capability, and on this model the cost is checkpoint-dependent. This release is the mild Pareto point. Measured on HumanEval+ (164 tasks, greedy,
enable_thinking=false), it is statistically indistinguishable from the unmodified model (0.396 vs 0.402 base pass@1). A more aggressive point from the same search (0/100 refusals instead of 15/100) scores 0.232, i.e. −25.6 pass@1 points. Do not assume a stronger abliteration of this model is equally usable; see the table below before choosing a different checkpoint.
One 12-trial Optuna search with Heretic 2.0.0 on a single Tesla T4 (fp16, batch 16), against
mlabonne/harmful_behaviors (bad prompts) and mlabonne/harmless_alpaca (good prompts), with the
default refusal keyword scorer and KL-divergence scorer. The uploaded checkpoint is Pareto front
index 1 (trial 10):
| ARA parameter | value |
|---|---|
| start / end layer | 8 / 15 |
| preserve_good_behavior_weight | 0.6514 |
| steer_bad_behavior_weight | 0.01234 |
| overcorrect_relative_weight | 0.6625 |
| neighbor_count | 1 |
| LoRA rank | 50 (48 matrices: attn.o_proj, conv.out_proj, mlp.down_proj × 24 layers) |
The ablation was merged into the bf16 weights, so this repo is a full model, not an adapter.
Refusals: Heretic's keyword scorer, 100 harmful_behaviors test prompts, greedy, ≤300 new tokens,
batch 16, CoT block stripped.
| model | refusals | HumanEval+ base / plus | NLL (20 SWE-bench q) |
|---|---|---|---|
| base coder distill | 90/100 | 0.402 / 0.384 | 2.6646 |
| this model (front idx 1) | 15/100 | 0.396 / 0.384 | 2.8517 (+7.0 %) |
| front idx 0 (0/100, KL 0.95) | 0/100 | 0.232 / 0.207 | 3.2637 (+22.5 %) |
Reading these honestly:
mlx_lm works with the repo id directly.bf16/: the full bf16 transformers checkpoint (3.48 GB, model.safetensors) with the custom
LFM3 modeling code, tokenizer and chat template. Load with trust_remote_code=True.coder-lfm3-1.7b-k3.1-pro-heretic-Q4_K_M.gguf — 1.1 GB, 5.05 bits/weight, for llama.cpp.
SHA-256 ede274e3fbdd2b8c825262000558ee90645b03e197c8b16ed1b4d4c024e85e58.Transformers (the custom modeling_lfm3.py is required):
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
mid = "AJ-Gonzalez/coder-lfm3-1.7b-k3.1-pro-heretic"
tok = AutoTokenizer.from_pretrained(mid, subfolder="bf16", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
mid, subfolder="bf16", trust_remote_code=True,
dtype=torch.bfloat16, device_map="auto")
messages = [{"role": "user", "content": "Write a Python function that merges two sorted lists."}]
prompt = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False,
enable_thinking=False)
inputs = tok(prompt, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=512, do_sample=False)
print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))
MLX (the repo root is a 4-bit MLX checkpoint, so the repo id works directly):
mlx_lm.generate --model AJ-Gonzalez/coder-lfm3-1.7b-k3.1-pro-heretic \
--prompt "Write a Python function that merges two sorted lists." --max-tokens 512
# measured on an M1 8 GB: 47 tok/s, 1.14 GB peak (48-token smoke)
llama.cpp with the Q4_K_M GGUF:
hf download AJ-Gonzalez/coder-lfm3-1.7b-k3.1-pro-heretic \
coder-lfm3-1.7b-k3.1-pro-heretic-Q4_K_M.gguf --local-dir .
llama-cli -m coder-lfm3-1.7b-k3.1-pro-heretic-Q4_K_M.gguf \
-p "Write a Python function that merges two sorted lists." \
-st -n 512 -t 8 \
--chat-template-kwargs '{"enable_thinking": false}'
# measured on an M1 8 GB, llama.cpp build 11960: 39.0 tok/s generation, 23.8 tok/s prompt
Notes on the llama.cpp line, from a verified run:
-st (single turn) is the chat path in current llama.cpp; -cnv is gone and errors with
invalid argument: -cnv.--chat-template-kwargs '{"enable_thinking": false}' closes the <think> block in the LFM3
template, so the answer starts immediately. Without it the model writes its scratchpad first and
a short -n can end before any code appears.--jinja is on by default in this build and uses the template embedded in the GGUF.The GGUF is reproducible from bf16/: convert_hf_to_gguf.py bf16 --outtype f16 (20 s) then
llama-quantize … Q4_K_M (25 s).
AJ-Gonzalez/coder-lfm3-1.7b-k3.1-pro
(a LoRA distill on the Kimi k3.1-pro coder corpus), which derives from
LiquidAI/LFM3-1.7B.LICENSE. Redistribution must keep it,
mark modified files (the ablation changed the weights only — both mlx-4bit/model.safetensors
and bf16/model.safetensors; the config, tokenizer and template are unchanged), and retain upstream notices.bf16/model.safetensors 3.5 GBmodel.safetensors 973 MBAJ-Gonzalez/coder-lfm3-1.7b-k3.1-pro