Ling-3.0-tiny — Pollard

Pollard shrank this model: 15.78 GB (f16) → 3.83 GB — 76% smaller, 4.1× down.

The smallest rung here; larger, higher-fidelity rungs are listed below.

format	this model's size
f16	15.78 GB
Q8_0	~8.36 GB
Q6_K	~6.47 GB
Q4_K_M	~4.58 GB
PollardMix (this repo's IQ3_S)	3.83 GB

Pollard builds of inclusionAI/Ling-3.0-tiny made with Pollard Weights — a ladder of measured-allocation quants (bits placed by per-layer sensitivity, not a uniform crush).

Standard GGUF — runs in stock llama.cpp / ik_llama.cpp, Ollama, LM Studio. Trellis (IQ*_KT) files need ik_llama.cpp; the K-quants run anywhere.

Model details

Which file should I choose?

Every rung is the same weights, sized to a different RAM budget by the measured allocation. Pick the largest one that fits your machine with room for context:

• ~8 GB RAM / VRAM → Q6_K (6.26 GB). Fits an ~11 GB box. Near-lossless — maximum quality.
• ~7 GB RAM / VRAM → IQ4_XS (4.64 GB). Fits an ~9 GB box. Higher fidelity — sensitive layers pushed to iq4_xs.
• ~6 GB RAM / VRAM → IQ3_S (3.83 GB). Fits an ~8 GB box. Beats same-size uniform IQ3 (table above). Recommended.

Available files

tok/s is hardware-specific; the machine it was measured on is stated in the errata.

Why this over a uniform quant

Held-out KL-divergence vs a Q6_K reference (lower = closer to the full model), measured on the same held-out set for every build:

At matched size the measured allocation sits below the uniform size↔KL curve. The measured mix: sensitive early layers get iq4_xs, most get iq3_s, the least-sensitive get iq2_s; every attention block stays q6_K/q5_K; embeddings/output stay q6_K; imatrix-uncovered MoE tensors are pinned so the aggressive base can't crash. (ffn sensitivity spread ~6×, attn spread ~16× across the 24 layers — that variance is exactly what a uniform quant wastes. The full per-tensor map is in Ling-3.0-tiny-Pollard.tensor-types.txt.)

Embed / output weights

Token-embedding and output tensors stay at q6_K, and every attention block is kept at q6_K/q5_K rather than dropped to the IQ base — measured sensitivity says those tensors don't tolerate crushing, so the bits are spent there and clawed back from the least-sensitive FFN experts.

Download a specific file

pip install -U "huggingface_hub[cli]"
hf download PollardWeights/Ling-3.0-tiny-Pollard \
  --include "Ling-3.0-tiny-Pollard-IQ3_S.gguf" --local-dir ./

How to run

These are standard GGUF and run with llama.cpp:

llama-server -hf PollardWeights/Ling-3.0-tiny-Pollard:IQ3_S

or from a local file:

llama-cli    -m Ling-3.0-tiny-Pollard-IQ3_S.gguf -ngl 99 -p "Explain why the sky is blue."
llama-server -m Ling-3.0-tiny-Pollard-IQ3_S.gguf -ngl 99      # OpenAI-compatible API + web UI at :8080

They also work in anything built on llama.cpp — LM Studio, koboldcpp, Jan, ramalama, Ollama (ollama run hf.co/PollardWeights/Ling-3.0-tiny-Pollard).

imatrix (calibration)

The importance matrix (Ling-3.0-tiny-Pollard.imatrix, included) was computed on a mixed-domain corpus so the matrix sees every register the model serves.

ARM / AVX

llama.cpp repacks weights into an interleaved layout at load time for faster inference on ARM and AVX machines — no special file needed, online repacking covers these quants. The old Q4_0_4_4/4_8/8_8 variants are not required.

Errata

• Trellis (IQ*_KT) quants need ik_llama.cpp to build/run; K-quants run in any recent llama.cpp.
• Measured allocation places bits by per-layer sensitivity under a size budget.
• Single machine; replication invited.

Credits & license

• Base model: inclusionAI/Ling-3.0-tiny
• Quantization tooling: llama.cpp (ggml-org)
• Method + tooling: Pollard Weights — measure first, no claim before a number.
• License: mit, inherited from the base model.

Built with Pollard Weights — frontier models, small hardware, no compromise.