NVFP4 quantization (4-bit floating point weights and activations) of Qwen3.8-27B-RFT-v1, for NVIDIA GPUs with native FP4 support. About 20 GB instead of 52 GB.
A model by Horizon Tech d.o.o., Sarajevo, Bosnia and Herzegovina.
| Developed by | Horizon Tech d.o.o., Sarajevo, Bosnia and Herzegovina |
| Model type | NVFP4 quantization of a 27B-parameter vision-language model |
| Quantized from | Qwen3.8-27B-RFT-v1 (fine-tune of Qwen/Qwen3.8-27B) |
| Format | NVFP4, 4-bit floating-point weights and activations (compressed-tensors nvfp4-pack-quantized) |
| Size | 20 GB (52 GB unquantized); MTP head in model_mtp.safetensors, full precision |
| License | Apache 2.0 |
| Released | November 2026 |
| Author | Hamza Abdagic (ORCID 0009-0002-4078-4922) |
| Contact | hamza.abdagic@horizon-tech.io |
| Repository | Format | Size | For |
|---|---|---|---|
| Qwen3.8-27B-RFT-v1 | bfloat16 safetensors | 52 GB | Transformers, vLLM, SGLang; further fine-tuning |
| Qwen3.8-27B-RFT-v1-NVFP4 | NVFP4 (4-bit) | 20 GB | vLLM on NVIDIA GPUs with FP4 support |
| Qwen3.8-27B-RFT-v1-mlx-4bit | MLX 4-bit | 14 GB | Apple silicon, text only |
QuantizationModifier(targets="Linear", scheme="NVFP4"), sequential pipeline per decoder layer.HuggingFaceH4/ultrachat_200k, up to 2,048 tokens each.lm_head, the vision tower, token embeddings, the in_proj_a / in_proj_b projections of the linear-attention layers, and the MTP head.model_mtp.safetensors) was retrained for RFT v1 by self-distillation and is kept in full precision. Measured on 120 held-out documents, the retrained head gets 3.15 drafted tokens accepted at 3 drafts and 4.06 at 5 drafts; the head copied from the base got 3.02 and 3.63. These are offline numbers, not serving speed.Serve with vLLM on a GPU with FP4 support:
vllm serve ghostrider0470/Qwen3.8-27B-RFT-v1-NVFP4
@misc{horizontech2026rftv1nvfp4,
author = {Abdagic, Hamza},
title = {Qwen3.8-27B-RFT-v1-NVFP4},
year = {2026},
publisher = {Hugging Face},
organization = {Horizon Tech d.o.o., Sarajevo},
howpublished = {\url{https://huggingface.co/ghostrider0470/Qwen3.8-27B-RFT-v1-NVFP4}}
}
Apache 2.0, as the base model.
model_backbone.safetensors 20 GBmodel_mtp.safetensors 849 MBghostrider0470/Qwen3.8-27B-RFT-v1