Pretype One
One model for typing, fixes and replies. It is a smaller Gemma 4 E2B (4-bit MLX) for Pretype, a macOS autocomplete app. One 1.5 GB download does four jobs:
- raw-text autocomplete (no chat template), with the weights in
model.safetensors; - typo fixes, replies and rewrites through the chat template, with the LoRA adapter in
adapters.safetensorsswitched on.
The adapter is not fused: a fuse into 4-bit weights erases it. Load it at run time. With the adapter off, the model is the autocomplete model.
How it was made
- Smaller token tables. mlx-community/gemma-4-e2b-4bit without the audio and vision towers. The vocabulary is cut from 262,144 to 82,000 tokens: the tokens most used in public subtitles, Wikipedia text and code. The tokenizer merges of a removed token are removed too, so its text splits into smaller tokens that stay. The file falls from 2.45 GiB to 1.46 GiB.
- Distillation. Gemma 4 E4B (8-bit) is the teacher, on 10 million tokens of public text in 20 languages (subtitles, Wikipedia, code, and messages that Gemma 4 E4B-it wrote for this purpose). The loss follows the teacher's top 16 tokens. Attention weights, per-layer gates and the down projections learn; the weights are 4-bit again at the end.
- Chat adapter. LoRA on
q_projandv_projof all 35 layers, rank 64, scale 20, 5500 steps, on the frozen 4-bit model. Data, 43,500 rows: typo fixes with synthetic typos on subtitle sentences, replies and rewrites that Gemma 4 E4B-it wrote on public subtitle text.
Sentences equal to public evaluation text were dropped. No user data is in any step.
Measurements
Pretype's evaluation harness, 2026-11-02 to 2026-11-12. Greedy decoding. The comparison is Gemma 4 E2B 4-bit for autocomplete and its instruct build for the chat jobs: two files, 2.5 GB + 3.0 GB.
| Comparison | This model | |
|---|---|---|
| Autocomplete: first word right, 6,000 rows, 20 languages | 21.2% | 24.2% |
| The same, only rows not met in the training text | 21.3% | 23.9% |
| Autocomplete: first word right, 5,649 rows of other sources | 20.8% | 22.8% |
| Fix: typo line restored exactly, 510 rows | 49% | 79% |
| Fix: change proposed on a clean line | 52 of 170 | 6 of 170 |
| Reply: answered, in the right language, 380 conversations | 96% | 96% |
| Reply: on topic, by a judge model, 380 conversations | 85% | 90% (p=0.04, trend) |
| Rewrite: answered, right language, right length, 400 rows | 74% | 86% |
| Peak memory of the process | 6.0 GiB | 2.1 GiB |
| Energy per suggestion, one session | 1048 mJ | 946 mJ |
| Energy per fix, one session | 2291 and 2474 mJ | 2224 mJ |
Limits
- Rewrites can come back unchanged. 34 of 400 rewrite requests gave the input text back. This occurs most where a small edit is correct.
- Rewrites are measured for form, not for sense. The test looks at the language and the length. No judge read the rewrites.
- Rewrites are slower than on the instruct build: 2.5 ms for each output character against 1.8 ms. The adapter is the cause.
- Rare tokens are gone. Text that needs a removed token costs more tokens. News text lost some accuracy from the cut before the distillation.
- Not a chat model. The adapter learned three short tasks. It was not tested on open questions.
- Energy differs by about 15% between sessions. The energy rows show "not more", not a gain.
Use
mlx-lm rejects the unused key/value projections of the KV-shared layers in strict mode, so load without it:
from pathlib import Path
from huggingface_hub import snapshot_download
from mlx_lm import generate
from mlx_lm.utils import load_model, load_tokenizer
path = Path(snapshot_download("nikimoriOne/Pretype-One"))
model, _ = load_model(path, strict=False)
tokenizer = load_tokenizer(path)
print(generate(model, tokenizer, prompt="The quick brown fox", max_tokens=20))
For the chat jobs, add the adapter (mlx_lm.tuner.utils.load_adapters(model, str(path))) and use
the chat template.
License: Apache-2.0, the license of Gemma 4.