Occamy-1.0 · NVFP4

Expert-only NVFP4 · mixed precision · 23.93 GB

Model collection · Checkpoint explorer · Project · Paper

Mixed-precision NVFP4 weights of Accio-Lab/occamy-1.0.

The routed MoE experts use NVFP4 weights and activations through NVIDIA Model Optimizer 0.44.0 NVFP4_EXPERTS_ONLY_CFG. Attention, routers, shared experts, embeddings, output head, and vision weights retain BF16. The original Occamy tokenizer and chat template are preserved. This checkpoint does not include a donor MTP head.

Calibration

128 records from the train split of CNN/DailyMail, configuration 3.0.0, truncated to 256 tokens. Sixteen separate test records were held out. The exact dataset revision and selection are in data-provenance.json; the quantization configuration is in quantization-recipe.json. No training or fine-tuning was performed.

For expert weights whose observers were not activated, ModelOpt used weight-derived or fused-tensor scaling during export. This limited calibration set does not establish coverage of every expert on every workload. The runtime checks below test the exported checkpoint.

Evaluation

Full scalar results are in VALIDATION.json. These are small regression checks, not general benchmark results. The add tool was executed locally and its result returned to the model; the weather case validates the structured call only. Code fences were removed for code execution; JSON checks require raw JSON. No broad vision, audio, video, long-context, or throughput claim is made. The validation context length was 2048.

Tested serving configuration

SGLang 0.5.13.post1, FlashInfer 0.6.12, PyTorch 2.11.0+cu130. Native NVFP4 execution requires compatible hardware and kernels.

python -m sglang.launch_server \
  --model-path Accio-Lab/occamy-1.0-NVFP4 \
  --quantization modelopt_fp4 --dtype bfloat16 \
  --moe-runner-backend flashinfer_cutlass \
  --attention-backend triton --mamba-backend triton \
  --mm-attention-backend sdpa \
  --tool-call-parser qwen3_coder --reasoning-parser qwen3 \
  --context-length 2048 --max-total-tokens 4096 \
  --max-running-requests 1 --max-mamba-cache-size 4 \
  --disable-cuda-graph --disable-radix-cache --disable-overlap-schedule

The tests used greedy decoding and chat_template_kwargs={"enable_thinking": false}.

vLLM compatibility

On a single H200, this checkpoint completed all 9 requests and passed all 9 semantic checks, covering text, code, strict JSON, tool calls and result follow-up, and images. The BF16 reference also passed 9/9. There were no request errors or truncated responses. This used the Marlin W4A16 path, not native FP4 compute. MTP was disabled. This is a small compatibility check, not a broad quality or speed benchmark.

Attribution and license

Original model: Accio-Lab, based on Qwen3.6-35B-A3B. Original Apache-2.0 license is included.

See Technical details for reproduction settings.

Occamy checkpoints

BF16 · GGUF · FP8 · NVFP4 · MLX 8-bit · MLX 6-bit · MLX 4-bit · MLX 3-bit · MTP head

Compare file sizes, validation scope and deployment commands in the checkpoint explorer.