pionus-0.5b-v0.2
A fine-tune of Qwen/Qwen2.5-Coder-0.5B-Instruct that writes git commit messages in the repository's style, for pionus, a CLI that runs a small model locally (Intel NPU, GPU or CPU). Version 0.2, while the training recipe still changes; it replaces nothing: pionus-0.5b-v0.1 stays as it is.
What changed from v0.1:
- The rules live in the weights. It is trained on pionus's prompt without the fixed rules (72 columns, imperative mood, the Conventional Commits format): the system prompt holds only what varies by repository (types and scopes used, recent subjects, a ticket, Portuguese or gitmoji when the repository uses them). The prompt is about 105 tokens shorter, so a message is about 15% faster on the CPU.
- Young repositories: a fifth of the examples have 0–9 earlier subjects; it copies an earlier subject half as often there.
- Release bumps whose version the diff adds are in the data (v0.1 never saw one).
Files
| File | What | SHA-256 |
|---|---|---|
model.safetensors (+ config and tokenizer) |
merged weights, bf16 | 3b58ebf9f8160089a8514bba9373be7f5d019032ce959cd254a7b5e1188e13a5 |
pionus-0.5b-v0.2-q4_k_m.gguf |
Q4_K_M for llama.cpp, 398 MB | 87b81896a02fc10e6ead34701e71ae19083bbc5f03738b0a45d73b3f2d7b824a |
Use with pionus
pionus model use pionus-0.5b-v0.2 # pionus 0.7.0 and later
pionus recognises this file by its SHA-256 and builds the shorter
prompt it was trained on. Used as any other GGUF file
(PIONUS_GGUF=...), or with pionus 0.6.0 and earlier, it gets
the full prompt, which it also handles, a little less well (see below). It
runs on pionus's llama.cpp devices (CPU and Vulkan GPUs); for the
OpenVINO devices (Intel NPU and GPU) there is no validated export yet.
Evaluation
eval-v1: 500 commits from the test split of CommitChronicle (repositories
never seen in training), 250 following Conventional Commits and 250
free-form; eval-short: 197 commits from young repositories (1–5 earlier
subjects). Greedy decoding, pionus at 94ababd (with the prompt this
model expects unless noted), Vulkan on an RTX 3050, with its retry and
file-list fallback.
| Model | type_match | type_macro_f1 | first_try | fallback | word_f1 | word_f1 (free) | young repos: fallback | young repos: first reply copied |
|---|---|---|---|---|---|---|---|---|
| Qwen2.5-Coder-0.5B-Instruct, upstream Q4_K_M | 0.164 | 0.123 | 0.384 | 0.388 | 0.135 | 0.098 | 0.340 | 0.340 |
| pionus-0.5b-v0.1 | 0.636 | 0.542 | 0.976 | 0.016 | 0.340 | 0.236 | 0.086 | 0.137 |
| pionus-0.5b-v0.2 | 0.632 | 0.526 | 0.972 | 0.012 | 0.337 | 0.236 | 0.046 | 0.076 |
| pionus-0.5b-v0.2, with the full prompt | 0.612 | 0.448 | 0.968 | 0.024 | 0.320 | 0.239 | 0.020 | 0.020 |
- type_match: same Conventional type as the real commit (of 250); type_macro_f1: the F1 of each type, averaged over the types.
- first_try: the first reply was usable; fallback: no reply was, and pionus wrote the message from the file list.
- word_f1: word overlap with the real subject, a rough proxy for content.
- With n = 250 a difference under about 0.06 is noise: on eval-v1, v0.1 and v0.2 are the same model; v0.2 is better on young repositories and faster.
CPU, Ryzen 5 5500 (6 cores), the first 50 commits of eval-v1, model load counted once: 0.98 s per message, against 1.15 s for v0.1. All the numbers and the method are in training/HYPOTHESES.md.
Training
- Data: 20,019 examples (20,001 trained, 18 too long) from the train
split of CommitChronicle
(revision
5fd076e): 15,600 commits with at least 10 earlier subjects, 4,000 from young repositories, 419 release bumps whose version the diff adds; at most 150 per repository, no cap per type. Prompts built by pionus itself with--rules omitted, the real subject as the target (examples file sha25623b9effc8b202e6d…). - Method: LoRA r=16, alpha=32, dropout 0.05 on all linear layers, lr 2e-4 cosine, warmup 3%, batch 4 × 4, 1 epoch, seed 0, loss on the reply only; 4-bit NF4 base with fp16 compute (QLoRA) on a Kaggle Tesla T4, 71 minutes. torch 2.11.0, transformers 5.20.1, peft 0.21.2.
- Export: merged into the bf16 base and quantized to Q4_K_M with llama.cpp
b11381 (
training/export_gguf.py --merged).
Training on a GPU is not bit-for-bit reproducible; generation is (greedy). The recipe, the data selection and the scripts are in training/ (bundle b3).
Limitations
- English messages only (the training data is English).
- It can invent details the diff does not show, and it does not know when the staged changes are several commits.
- It tells
fixfromfeatno better than v0.1: the training labels are the authors' own, and they disagree on that boundary. - Untested in repositories whose recent subjects are not in English.
Licence and attribution
The weights are released under Apache-2.0, like the base model.
CommitChronicle (JetBrains Research) has no single licence: it gathers
commits from repositories under MIT, Apache-2.0 and BSD-3-Clause, and asks
that uses follow their terms, attribution included. The dataset records the
repository and licence of every commit (repo and license fields), which is
the provenance of the training data.