Apex-35B-V1.0-MXFP4-MoE-GGUF
Apex-35B-V1.0 (
Apexintelligence-AI/Apex-35B-V1.0) is an Apache-2.0 open-weight 35B sparse Mixture-of-Experts (MoE) model with approximately 3B active parameters and a 262K context window, built onQwen/Qwen3.6-35B-A3B. It targets general reasoning, scientific research, CFD, robotics simulation, and agentic coding, with Claude Code support. Its post-training combines proprietary research trajectories, recursive self-improvement (RSI), dynamic thinking, code execution, and iterative refinement. Key benchmarks include SciCode 62.6/75.6, FrontierScience 40.0, PaperBench 60.9, BrowseComp 69.4, BFCL V4 84.0, Terminal-Bench 2.1 65.2, SWE-bench Pro v2 92.4, SWE-bench Verified No Internet 73.4, LiveCodeBench v6 82.3, MMLU-Pro 87.2, HLE 47.2, and GPQA Diamond 83.3. It also scores 44.5 on RoboRSI and 21.6 on Real2Sim, with a 79/0/21 win/draw/loss record against its base model on CFD-Bench. Available in BF16 and GGUF quantizations (Q8_0,Q6_K,Q5_K_M, andQ4_K_M), it supports Transformers,llama.cpp, vLLM, SGLang, TokenSpeed, KTransformers, and Unsloth, including XML tool calling and Multi-Token Prediction (MTP).
[!NOTE] MTP — Speculative Decoding: Yes
[!NOTE] Multi-Token Prediction (MTP) GGUF is a specialized GGUF model file format extension that integrates speculative decoding directly into the model weights to significantly accelerate local inference. Unlike traditional speculative decoding which requires a separate, smaller "draft" model, MTP GGUF files include additional output heads within the main model architecture that predict multiple future tokens in a single forward pass.
Model Files
| File Name | Quant Type | File Size | File Link | Description |
|---|---|---|---|---|
| Apex-35B-V1.0.BF16.gguf | BF16 | 71.1 GB | Link | Full BF16 weights. Highest quality, largest file size. |
| Apex-35B-V1.0.MXFP4_MOE.gguf | MXFP4_MOE | 20.3 GB | Link | MXFP4_MOE: Highly optimized for NVIDIA Blackwell hardware accelerators (e.g., RTX Pro 6000, RTX 50-series, B200, B300), offering native hardware-accelerated FP4 Tensor Core speeds and a strong balance of size and performance. |
| Apex-35B-V1.0.Q3_K_L.gguf | Q3_K_L | 18.6 GB | Link | Lower quality but usable, good for low RAM availability. |
| Apex-35B-V1.0.Q5_K_M.gguf | Q5_K_M | 25.3 GB | Link | High quality, recommended. |
| Apex-35B-V1.0.Q6_K.gguf | Q6_K | 29.2 GB | Link | Very high quality, near perfect, recommended. |
| Apex-35B-V1.0.Q8_0.gguf | Q8_0 | 37.8 GB | Link | Extremely high quality, generally unneeded but max available standard quant. |
| Apex-35B-V1.0.mmproj-bf16.gguf | mmproj-bf16 | 903 MB | Link | Multimodal projection file in BF16 format. Used for vision/language models. |
MXFP4_MOE Test Run
Tested Apex-35B-V1.0.MXFP4_MOE.gguf on an NVIDIA RTX PRO 6000 Blackwell Server Edition GPU with 96 GB of VRAM, using llama-cli with full GPU offloading and MTP speculative decoding.
Performance Metrics
| Metric | Value |
|---|---|
| Average Prompt Processing Speed | 213.94 tokens/s |
| Average Generation Speed | 293.37 tokens/s |
| Total Wall Time | 46.6 seconds |
| Context Length | 16,384 tokens |
| KV Cache (K) | q8_0 |
| GPU | NVIDIA RTX PRO 6000 Blackwell Server Edition |
| GPU VRAM Capacity | 96 GB |
| Peak GPU VRAM Usage | 19.86 GiB |
| Peak GPU Utilization | 81% |
| GPU Layers Offloaded | 99 (-ngl 99) |
| Random Seed | 42 |
| Speculative Decoding | MTP (draft_n_max=2) |
| Test Runs | 5 |
Test Shots
| Shot | Test |
|---|---|
| Shot 1 | C: Insert Node at End of Singly Linked List |
| Shot 2 | Jogger and Train Passing Time |
| Shot 3 | Two Pipes Filling a Tank |
| Shot 4 | Students in Examination Rooms |
| Shot 5 | Father and Son Ages |
Run Inference
./build/bin/llama-cli \
-m /path/to/models/Apex-35B-V1.0.MXFP4_MOE.gguf \
--spec-type draft-mtp \
--spec-draft-n-max 2 \
-ngl 99 \
-c 16384 \
--cache-type-k q8_0 \
--seed 42 \
-n 8192 \
-st \
-p "Your prompt here"
Performance results reflect this specific hardware and inference configuration. Actual throughput may vary depending on the prompt, model, software build, and runtime settings.
llama.cpp
LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp
Snapshots (5)
Quantized using llama.cpp by prithivMLmods