[!IMPORTANT]

W8A8 block-128 FP8 quant of migtissera/Tess-4-27B

Runtime: standard vLLM fp8 quant_method (e4m3, weight_block_size [128, 128], dynamic activations). All evaluation below was run on tcclaviger/vllm:latest on RDNA 4 (gfx12xx). Not validated on any other hardware at this time.

Tess-FP8

Block-wise W8A8 FP8 (e4m3fn) quantization of Tess-4-27B by Migel Tissera — an agentic, thinking-native finetune of Qwen3.6-27B. Sibling quants: Tess-27B-RFI (int8+4-bit hybrid) and Tess-27B-RFA (all-attention 4-bit). All credit for the model to its author; this repo only changes the numerics.

Quantization by component

• All attention and MLP linear weights (self_attn q/k/v/o, MLP gate/up/down, GDN linear-attention in_proj_qkv / in_proj_z / out_proj) — FP8 e4m3fn, block-wise 128×128, per-block bf16 dequant scales (weight_scale_inv), dynamic activation scheme (W8A8).
• MTP speculative-decode head — its MLP and attention projections are FP8 by the same scheme; its fc and norms stay bf16.
• Kept in bf16 (not quantized) — vision encoder, GDN A_log/conv1d/dt_bias/in_proj_a/in_proj_b, embeddings, norms, and the lm_head.

Tensor layout and naming match the Qwen3.6-27B-FP8 production reference exactly.

Serving context — 512K via YaRN

All evaluation below was run while serving at --max-model-len 524288 (512K tokens), extended from the native 256K window with YaRN via --hf-overrides:

{"text_config": {"rope_parameters": {"rope_type": "yarn", "factor": 2.0,
 "original_max_position_embeddings": 262144, "mrope_interleaved": true,
 "mrope_section": [11, 11, 10], "partial_rotary_factor": 0.25,
 "rope_theta": 10000000}}}

Evaluation results

Six builds measured with identical methodology, each against its own live vLLM endpoint (July 2026): Qwen3.6-27B (bf16 base) → Tess-4-27B (the tune, bf16) → Tess-27B-RFI (int8+4-bit hybrid) → Tess-27B-RFA (all-4-bit) → Tess-FP8 (this build, W8A8 block-128 FP8), with Qwen3.6-35B-A3B (MoE, bf16) as a comparative reference. Bold marks the best score in each row (ties all bolded).

Quantization cost (W8A8 FP8 block-128) — Tess-4-27B → Tess-FP8

Quality

Multiple-choice accuracy is lm-eval loglikelihood scoring, 0-shot.

Long-context positional recall (codeneedle)

Verbatim function recall under 10K–80K-token contexts.

Tool calling & accuracy benches

Decode throughput — tok/s output (ISL 128 / ISL 512)

vllm bench serve, random dataset, OSL 128, saturation, 4× R9700 (gfx1201), TP 4.

MTP draft acceptance by work category

Measured from live serving logs, k=5 draft tokens, drafted-token-weighted aggregation.

Notes

All builds serve on the tcclaviger/vllm:latest image, which has kernel tunes baked in. TunableOp is untuned — GEMMs run on default heuristic-determined values. Base Qwen3.6-27B figures are the 2026-07-12 re-measurement on the same tcclaviger/vllm:latest image and thinking-OFF methodology as every other build, replacing an earlier non-comparable run.

Credits

• Tess-4-27B by Migel Tissera (migtissera/Tess-4-27B) — the model quantized here:

  @misc{tissera2026tess4,
    title        = {Tess-4-27B},
    author       = {Migel Tissera},
    year         = {2026},
    howpublished = {\url{https://huggingface.co/migtissera/Tess-4-27B}}
  }
  

• codeneedle (positional recall) originally by Alexander Ziskind, expanded test suite by tcclaviger (Rob Smith).

• Tool-calling scenarios (incl. TC-61) run on tool-eval-bench by SeraphimSerapis (Tim Messerschmidt), scenario methodology adapted from ToolCall-15 by stevibe.

• GSM8K, MMLU, and IFEval run via tool-eval-bench's built-in accuracy benchmarks at their defaults: GSM8K 8-shot CoT, MMLU 5-shot, IFEval zero-shot.