APEX MTP Vision Apache-2.0

Agents-A1-MTP-APEX

English | 📖 中文文档

35B agentic MoE that reaches trillion-parameter performance · APEX-quantized GGUFs + BF16 + mmproj

🤖 About Agents-A1

Agents-A1 is a 35B-parameter Mixture-of-Experts agentic model from InternScience, post-trained on top of Qwen3.5-35B-A3B via a three-stage paradigm: full-domain SFT → domain-level teacher training → multi-teacher multi-domain on-policy distillation.

Despite operating in the ~35B model class, Agents-A1 delivers highly competitive performance against frontier-scale systems such as GPT-5.5, DeepSeek-V4-pro, and Kimi-K2.6 — achieving SOTA on Seal-0 (56.4), HiPhO (46.4), FrontierScience-Olympiad (79.0), IFBench (80.6), IFEval (94.8), and best-among-comparable on BrowseComp (75.5), XBench-DS-2510 (86.0), GAIA (96.0), SciCode (44.3), HLE (47.6), and MolBench-bind (56.8).

This GGUF package includes the mmproj-F16.gguf vision projector for multimodal (image + text) capabilities with llama.cpp. MTP layers are extracted from Qwen3.5-35B-A3B and injected into Agents-A1's safetensors (see MTP Extraction & Injection section). License: Apache-2.0.

🧠 Model Details 🔧 MTP Extraction & Injection

The released InternScience/Agents-A1 checkpoint is a 40-layer Qwen3.5-35B-A3B MoE without MTP (Multi-Token Prediction) layers. To enable MTP acceleration in llama.cpp (which speeds up long-context generation by 10–30%), we extract the 1 MTP layer from Qwen3.5-35B-A3B and inject it into Agents-A1's safetensors before GGUF conversion.

The full pipeline has 4 steps: (1) Extract 785 MTP tensors from Qwen3.5-35B-A3B (filter keys containing "mtp"); (2) append as a new shard model-15-of-15.safetensors and update model.safetensors.index.json's metadata.total_size and weight_map (do not modify the original 14 shards); (3) convert to BF16 GGUF via the master-branch llama.cpp convert_hf_to_gguf.py — the master build auto-detects regular layers blk.0–39 + MTP layer blk.40.nextn.* (785 tensors); (4) quantize with APEX llama-quantize using qwen36_35b_mtp_*.txt configs which already include the blk.40 override (MTP kept at Q8_0 across all tiers — no manual patching needed). The imatrix Qwen3.5-35B-A3B.imatrix.gguf is reused directly (same architecture, compatible weights).

Reproduction command (example: I-Compact tier)

F:\llama.cpp\...\llama-quantize.exe ^
  --imatrix J:\Models\Qwen3.5-35B-A3B.imatrix.gguf ^
  --tensor-type-file E:\apex-quant\configs\qwen36_35b_mtp_compact.txt ^
  J:\Models\Agents-A1-MTP-GGUF\Agents-A1-MTP-BF16.gguf ^
  J:\Models\Agents-A1-MTP-GGUF\Agents-A1-MTP-APEX-I-Compact.gguf ^
  Q4_K_M

📊 BenchLocal Results (APEX-I-Compact, 16.14 GB)

RTX 5070 Ti 16GB + 128GB RAM · No-thinking mode achieves higher ceiling (BugFind +12) but suffers more retries on complex agent scenarios.

🚀 Usage

llama.cpp (text only)

hf download SC117/Agents-A1-MTP-APEX-GGUF --include "*.gguf" --local-dir ./models
./llama-server -m ./models/Agents-A1-MTP-APEX-I-Compact.gguf -ngl 99 -c 131072

llama.cpp (vision + text)

./llama-server -m ./models/Agents-A1-MTP-APEX-I-Compact.gguf --mmproj ./models/mmproj-F16.gguf -ngl 99 -c 131072

vLLM

vllm serve SC117/Agents-A1-MTP-APEX-GGUF --port 8000 --tensor-parallel-size 1 --max-model-len 262144 --reasoning-parser qwen3
·
Tool-call variant
vllm serve SC117/Agents-A1-MTP-APEX-GGUF --port 8000 --tensor-parallel-size 1 --max-model-len 262144 --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coder

SGLang

python3 -m sglang.launch_server --model-path "SC117/Agents-A1-MTP-APEX-GGUF" --host 0.0.0.0 --port 30000

🎛️ Recommended Sampling Parameters

From the official Agents-A1 model card:

💡 What is APEX?

These GGUF files are quantized using APEX, an MoE-aware mixed-precision quantization technique. APEX classifies every tensor by its role — routed expert, shared expert, SSM, or attention — and applies a layer-wise precision gradient, giving sensitive edge layers (including the MTP layer) higher precision and compressing redundant middle layers more aggressively.

APEX beats Q8_0 perplexity at half the size — and even beats F16 in some cases.

The qwen36_35b_mtp_*.txt configs include overrides for blk.40 (the MTP layer), preserving it at Q8_0 across all four I- tiers. The same Qwen3.5-35B-A3B.imatrix.gguf is reused (same architecture, compatible MoE expert layout).

📦 APEX Quantization Tiers

BF16 source: Agents-A1-MTP-BF16.gguf (66.19 GB). imatrix: Qwen3.5-35B-A3B.imatrix.gguf (reused from base model).

Links

• Original Model: https://huggingface.co/InternScience/Agents-A1
• Base Model (MTP source): https://huggingface.co/Qwen/Qwen3.5-35B-A3B
• Paper: https://arxiv.org/abs/2606.30616
• APEX Quantization: https://github.com/mudler/apex-quant
• BenchLocal Results: https://scorp1o117.github.io/benchlocal-results/

Citation

@misc{bai2026scalinghorizonparametersreaching,
      title={Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent},
      author={Lei Bai and Zongsheng Cao and Yang Chen and Zhiyao Cui and Shangheng Du and Yue Fan and Shiyang Feng and Zijie Guo and Haonan He and Liang He and Xiaohan He and Shuyue Hu and Yusong Hu and Songtao Huang and Yichen Jiang and Hao Li and Xin Li and Dahua Lin and Weihao Lin and Fenghua Ling and Dongrui Liu and Zhuo Liu and Runmin Ma and Chunjiang Mu and others},
      year={2026},
      eprint={2606.30616},
      archivePrefix={arXiv}
}