🪐 Qwopus3.8-27B-Flash (W4A16)

[!NOTE] Qwopus3.8-27B-Flash W4A16 is a 4-bit weight / 16-bit activation quantized version of Qwopus3.8-27B-Flash, using AutoRound with GPTQ group size 128. It preserves the Flash model's efficiency characteristics while reducing memory footprint.

[!TIP] W4A16 Quantization · ~65% Smaller · Same Efficiency Characteristics · FP16 Parity on ARC-Easy

📊 Benchmark Results

ARC-Easy

Model	Accuracy	Acc Norm
Qwopus3.8-27B-Flash (FP16)	83.71%	77.06%
Qwopus3.8-27B-Flash (W4A16)	84.13%	78.54%

The W4A16 quantized version matches or slightly exceeds the FP16 original on ARC-Easy.

[!NOTE] Additional benchmarks (MMLU, HumanEval, BBH, WikiText2 PPL) are in progress. Results will be added to this README when complete.

💾 Quantization Details

Parameter	Value
Algorithm	AutoRound + GPTQ
Weight bits	4
Activation bits	16
Group size	128
Version	AutoRound 0.12.3

Model size: ~19GB (vs ~54GB FP16) — 65% reduction

The quantization preserves the following layers in FP16 for quality:

Linear attention projection matrices (in_proj_a, in_proj_b) across all 64 layers
MTP (Multi-Token Prediction) module
Visual encoder

🚀 Usage

Works with any framework supporting GPTQ/AutoRound checkpoints:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "Jackrong/Qwopus3.8-27B-Flash-W4A16/Jackrong_Qwopus3.8-27B-Flash-w4g128"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype="auto"
)

Or via vLLM for high-throughput serving:

vllm serve Jackrong/Qwopus3.8-27B-Flash-W4A16/Jackrong_Qwopus3.8-27B-Flash-w4g128 \
    --dtype auto --tensor-parallel-size 1

🎯 Recommended Use Cases

Local MTP speculative decoding where decoder throughput and draft acceptance matter.
Long-running agent workflows with repeated tool calls and iterative edits.
Resource-conscious workloads where wall-clock time and generated-token cost are operational constraints.
Memory-constrained environments where the 65% size reduction enables local deployment on smaller GPUs.

⚠️ Limitations

Accuracy–Efficiency Trade-off: The underlying Flash fine-tuning prioritizes practical inference efficiency. On the reported 2,500-question MMLU-Pro mixed set, Qwopus scores 91.28% versus 92.73% for the Qwen3.8 base comparison.
Reasoning Stability: Although the long-tail behavior is improved in the reported evaluation, edge cases may still exhibit reasoning drift, loops, or inefficient trajectories.
Agent Dependence: Tool-use quality and task completion remain dependent on the prompt, tool environment, orchestration layer, and feedback loop.
Experimental Release: This is an independent experimental release for research, local evaluation, and technical exploration.

🙏 Acknowledgements

Special thanks to:

Qwen for the Qwen3.8-27B base-model foundation.
Jackrong for the Qwopus3.8-27B-Flash fine-tuning.
Unsloth for practical, efficient large-model fine-tuning tooling.
The open-source community for training tools, evaluation methods, and technical discussion.

JC1DA/Qwopus3.8-27B-Flash-INT4-W4A16