Tess-4-27B · NVFP4

NVFP4 (4-bit floating point) quantization of migtissera/Tess-4-27B — 19 GB (vs 52 GB BF16, −63%). Blackwell-native W4A4 for maximum speed on RTX 50-series / B200; runs on Hopper via vLLM fallback.

Produced with llm-compressor (scheme=NVFP4, group 16, FP8 block scales), calibrated on 512 of Tess-4's own on-policy generations (reasoning + coding + tool-call traces) rather than generic text. Vision tower, lm_head, and MTP block kept in BF16.

Quality check

Verified via vLLM (greedy): factual recall, multi-step reasoning (correct algebra on classic trap questions), and code generation with Tess-4's characteristic verification-style thinking all intact — including weight-scaled reasoning (empty think block on trivial questions).

Usage (vLLM)

vllm serve migtissera/Tess-4-27B-NVFP4 --trust-remote-code

On hybrid-arch builds you may need --max-num-seqs 64 (mamba cache blocks). For agentic/tool use add: --enable-auto-tool-choice --tool-call-parser hermes --reasoning-parser qwen3.

Pro tip: pair with the EAGLE-3 draft for compound speedup — quantization and speculative decoding stack.

License

Apache 2.0, matching the base model. Part of the Tess series by Migel Tissera.