HobbyLM-1B Instruct for MLX

Instruct (Transformers) | Base | GGUF | Online Demo | GitHub

This is the tested Apple MLX build of HobbyLM-1B Instruct. It uses partial 8-bit quantization and remains mostly FP32. Only the routed-expert down-projections in transformer layers 10–19 use weight-only affine 8-bit quantization. Earlier experts, other expert projections, the MoE router, attention, embeddings, shared expert and output path remain FP32. Router weights and calculations remain FP32.

Install and run

Requirements: an Apple-silicon Mac and Python 3.10–3.13.

python3.12 -m venv .venv-hobbylm
source .venv-hobbylm/bin/activate
pip install "huggingface-hub==1.33.0"
hf download harims95/hobbylm-1B-mlx \
  --local-dir hobbylm-1B-mlx
pip install ./hobbylm-1B-mlx/runtime

hobbylm-mlx \
  --model ./hobbylm-1B-mlx \
  --prompt "Explain sparse routing in one sentence." \
  --temperature 0 \
  --repetition-penalty 1 \
  --max-tokens 32

For interactive chat, omit --prompt:

hobbylm-mlx --model ./hobbylm-1B-mlx

For exact reproduction, add --revision f77a183c4015e879b2bf28f78e48e7a31aac210f (the first published revision) to the download. The installed runtime also defaults to harims95/hobbylm-1B-mlx, so hobbylm-mlx --prompt "Your question" downloads the model automatically. Generation runs locally after the initial download.

The CLI uses the released SFT chat template and adds no system message unless --system is explicitly supplied.

What “partial 8-bit, mostly FP32” means

The frozen 30-prompt suite generated 2,359 FP32 tokens in 43.76 seconds and 2,363 partial-8-bit tokens in 42.45 seconds. That is approximately 3% less elapsed time in this run. It is not a general hardware-independent speedup claim. Peak MLX memory in the deterministic benchmark was 3.53 GiB.

The source model was not trained or fine-tuned during conversion.

Source and checksums

The private conversion source contains the same model weights as the public Instruct repository. See SHA256SUMS.txt for every published file and quantization-manifest.json for machine-readable conversion metadata.

Tested environment

The tokenizer files are byte-identical to those of the public harims95/hobbylm-1B-instruct. Its template uses <|user|> and <|assistant|> role markers and EOS token 50256. Single-turn and multi-turn prompt IDs were checked against that immutable tokenizer.

Validation

• Strict loading completed without missing or unexpected tensors.
• All 19 sparse MoE routers remained FP32.
• Both backends received identical prompt IDs in all 30 frozen cases.
• The partial-8-bit build produced 30 non-empty outputs.
• 26 of 30 output token sequences exactly matched FP32; four differed.
• Manual review found the four changed answers equivalent, with zero FP32-pass to partial-8-bit-fail regressions.
• Expert-set agreement was 99.75% while both backends received identical tokens.
• The deterministic benchmark measured 27.0 generated tokens/s and 3.53 GiB peak MLX memory.

Machine-readable evidence is under validation/. This build inherits the quality and safety limitations of the source checkpoint and is not intended for high-stakes use.

License

Apache License 2.0 for the weights and the runtime code in this repository. The model architecture and original model code are by Harish (github.com/harishsg993010/HobbyLM), released under Apache-2.0 with his permission. The GPT-2 tokenizer files are MIT (OpenAI). Training data keeps its own terms; attributions and the open training-data licence question are in the NOTICE of hobbylm-1B and its training-data licence notes. MLX and the other Python packages keep their own licences.

Credits

Built by Hariharan and Prabhurajhan at Fuel Labs. Architecture based on HobbyLM by Harish. MLX conversion and validation by Prabhurajhan.