Winnow-E4B

Winnow-E4B is a compact EldanRing fine-tune of Gemma 4 E4B IT for typed decisions, chat, and image input. Give it a shared state and questions with known answer options; it scores the candidates without generating an explanation. Typical uses include routing requests, choosing actions, checking conditions, and rating urgency.

The GGUF downloads contain the merged fine-tune. No separate adapter or base-model download is needed. Image input uses the matching vision projector.

Quickstart · Inference code · API reference · Benchmarks

Decision results

These published Q8 results compare the models on the same questions. Jev and Kev report top-choice accuracy; Typed reports agreement with synthetic teacher labels.

Winnow-12B leads on the public Jev and Kev suites. E4B leads by 2.2 points on the local Typed teacher-agreement panel. These frozen panels were used during development. JevBench reports public-subset accuracy rather than the composite leaderboard score.

The 12B Kev-clean comparator is 852/1,046, distinct from its separate 853/1,046 release campaign. Evaluation details cover uncertainty, calibration, and scope.

Downloads

Choose Q8 for a smaller memory footprint, or BF16 for higher-precision weights. Add the matching projector for images.

Use an exact target filename or a Winnow preset. Checksums identify every file.

Running the model

The quickstart covers installation, downloads, and launch commands. The server provides /v1/systemone for decisions and /v1/chat/completions for ordinary chat, streaming, and images.

F16 is the default and recommended target K/V cache. Cache precision is separate from the GGUF weight format; use --cache q8_0 for an explicit Q8 override. The pinned MTP assistant uses the target's shared K/V cache.

Questions share a state prefill and can reuse a cached prefix. Candidate probabilities are conditional on the supplied options; confidence measures concentration, not guaranteed correctness.

Measured performance

On an RTX 5070 Ti 16 GB, the matched 8K text workload delivered 147.8 decisions/s for E4B Q8 versus 67.2 decisions/s for 12B Q8 at 64 questions per request. These are medians of ten warm requests with Q8 weights and KV cache; results depend on workload and batching.

Capacity suggestions are inferred from measurements. Six cached questions over a 65,023-position image state took 129–131 ms after an 11.59 s first request. A separate 62,431-position multimodal prompt generated 512 tokens at 71.2 tokens/s. See timing and memory details.

Calibration and adaptive reasoning

For the direct text profile, Q8's fitted winnow.temperature is 1.2574172017327816, based on 778 calibration questions. BF16's separately fitted value is 1.3331553765162731. Temperature changes probabilities without changing the winning option; the values belong to their respective runtime profiles.

Direct decisions are the default. Adaptive reasoning adds generated context before rescoring a question. A calibrated 50:50 evaluation improved Brier 0.524807 → 0.481402 and NLL 1.591870 → 1.385153, while source-equal agreement changed +0.083 pp. This profile combines calibration with generated context.

A separate historical uncalibrated recipe gained on Jev and Kev but lost 57 Typed teacher agreements (1,447 → 1,390/2,000), for a net loss of five across all 3,277 decisions. Mean direct HTTP time inside that workflow was 37 ms; the full router averaged 234 ms, including bypasses. Reasoning evaluations give the policies and timing definitions.

MTP separately drafts ordinary chat tokens with the matching E4B assistant. The 8K Q8 vision+MTP4 preset sampled 10,197 MiB peak memory. Direct serving needs no assistant. See runtime profiles and assistant attribution.

Training and limitations

Winnow-E4B is a rank-32, alpha-64 LoRA fine-tune. Language tensors were trained; vision and audio modules remained frozen. The adapter was merged in FP32 before Q8_0 and BF16 export. The private training set combines contrastive synthetic decisions, verified labels, teacher distributions, semantic tasks, and targeted hard cases. Training, development, calibration, and reserved tests were separated; the training data and pipeline are not released.

Credits and license

Winnow-E4B is an independent fine-tune by EldanRing of Google DeepMind's Gemma 4 E4B IT, released under Apache 2.0. See LICENSE and NOTICE.

The separate inference code builds on llama.cpp and preserves its MIT license. Jev-style describes the typed-decision interface; Winnow is not affiliated with or endorsed by TypeSafe, Google, or llama.cpp.