Millie-1.1-35B-A3B-Ternary

Millie 1.1 is a language and vision model from LLMs For All, built for local inference on memory-constrained devices. It has approximately 35 billion total parameters and 3 billion active parameters per token. Roughly 90% of its weights use a ternary representation; the remaining tensors use higher precision.

These are the weights used by the flagship model in the initial release of the Millie iPhone app.

Files

File	Purpose	Download size
`Millie-1.1-35B-A3B-Ternary.gguf`	Language model, tokenizer, and chat template	8.31 GB
`Millie-1.1-35B-A3B-vision.gguf`	Matching vision encoder/projector for image input	0.30 GB
`inference-settings.json`	Thinking and non-thinking sampling defaults	—
`chat_template.jinja`	A readable copy of the template embedded in the GGUF	—
`SHA256SUMS`	Checksums for the released GGUF files	—

Sizes are decimal GB. Disk size is not runtime RAM usage. The vision file is optional for text-only use.

Run Millie

Download the archive for your platform from Millie Runtime v0.1.0 and extract it.
Download Millie-1.1-35B-A3B-Ternary.gguf from this repository.
Open a terminal in the extracted runtime folder and run:

./bin/llama-cli \
  -m /path/to/Millie-1.1-35B-A3B-Ternary.gguf \
  --jinja --reasoning off \
  --temp 0.7 --top-p 0.8 --top-k 20 --min-p 0 \
  --presence-penalty 1.5 --repeat-penalty 1

Replace /path/to/ with the folder where you downloaded the model, then type a message to chat.

For images, also download Millie-1.1-35B-A3B-vision.gguf and add:

--mmproj /path/to/Millie-1.1-35B-A3B-vision.gguf --image /path/to/photo.jpg

The runtime supports ordinary inference and a low-memory mode that keeps a bounded number of experts resident per layer. Low-memory mode currently requires Apple Metal. See low-memory setup.

Generation settings

Millie inherits Qwen's sampling defaults for non-thinking and thinking generation:

Mode	Temperature	Top-p	Top-k
Non-thinking	0.7	0.8	20
Thinking	1.0	0.95	20

Both use min-p 0, presence penalty 1.5, and repetition penalty 1.0. For thinking, use --reasoning on --temp 1.0 --top-p 0.95 instead.

Evaluation

We compare the Qwen 3.6 teacher, the compressed model with unrestricted expert routing, and the compressed model in a low-memory configuration.

Benchmark	Qwen 3.6 35B-A3B	Millie unrestricted	Millie low-memory
MMLU Redux, clean question set	93.94%	91.62%	91.68%
GPQA Diamond	86.36%	75.76%	74.75%
HumanEval+	93.29%	90.24%	91.46%
IFBench, loose	68.33%	62.00%	60.33%
IFBench, strict	63.67%	60.00%	59.00%
MuSR	66.93%	64.15%	66.40%

The low-memory configuration uses 64 hot experts per layer, a +1 routing-logit bias for hot experts, and a simulated 8-bit (Q8_0) KV cache. The evaluations used single sampled runs with up to 64K generated tokens per example.

Attribution and license