中文Sign up

rapid-mlx/Qwen3.8-27B-4bit-MTP-fp16-MLX

Source: 🤗text-generationapache-2.027.4B params55 GBsafetensors✓ 5 checksumsupdated 7d ago
or submit a torrent

Run it on your Mac

curl -fsSL https://rapidmlx.com/install.sh | bash
rapid-mlx serve qwen3.8-27b-4bit-fp16
curl -s http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"default","messages":[{"role":"user","content":"Say hello"}]}'

Use it from OpenCode / Claude Code / Codex: rapid-mlx agents <name> --setup · Agent setup

Download the Desktop app · 32 GB unified memory recommended for the 16.3 GB target and MTP weights.

Qwen3.8-27B 4-bit MLX + MTP — FP16 sibling

This is the M1/M2-oriented FP16 sibling of rapid-mlx/Qwen3.8-27B-4bit-MTP-MLX. It preserves the 4-bit target weights and colocated native Multi-Token Prediction (MTP) sidecar, while storing every non-quantized BF16 tensor as FP16. M1 and M2 chips do not accelerate BF16 matrix multiplication; M3 and newer Macs should normally use the original BF16 checkpoint.

Rapid-MLX

rapid-mlx serve qwen3.8-27b-4bit-fp16 --no-think

Or use the repository ID directly:

rapid-mlx serve rapid-mlx/Qwen3.8-27B-4bit-MTP-fp16-MLX --no-think

--no-spec-decode remains available when a workload does not benefit from MTP. Rapid does not silently substitute this checkpoint for the regular qwen3.8-27b-4bit alias.

Conversion and validation

  • Source revision: aa985c29ff5b334cbfdcbbc787d47e66e9d9e456
  • 1,705 BF16 tensors converted to FP16.
  • 506 non-BF16 tensors preserved with identical names, shapes, dtypes, and values, including all packed U32 quantized weights.
  • All four safetensors files were read back and verified; the MTP sidecar loaded all 31/31 expected tensors in Rapid-MLX.
  • On an M3 Ultra, two of three fixed greedy prompts were byte-identical to the BF16 source. The third differed by one semantically equivalent phrase. This is a normal FP16/BF16 numerical boundary, so universal byte identity is not claimed.
  • On the FP16 target, greedy output with MTP enabled was byte-identical to its autoregressive output in the paired smoke test.

The machine-readable fp16-conversion.json records the conversion contract and counts. M1/M2 prefill measurements will be added after the paired hardware qualification completes.

Contents

  • model-*.safetensors: Qwen3.8-27B MLX 4-bit target weights with FP16 non-quantized tensors
  • mtp/model.safetensors: matching 4-bit native MTP drafter with FP16 non-quantized tensors
  • tokenizer, processor, generation, and model configuration files

Upstream model licenses and limitations continue to apply.