---

🤗 Model   |    🔀 OpenRouter   |    💻 Github   |    🧭 ModelScope   |    🚀 Nex-AGI

Nex-N2-Mini-NVFP4

NVFP4 (4-bit) quantized build of Nex-N2-mini — an agentic model with Agentic Thinking.

This repository contains a post-training-quantized checkpoint of nex-agi/Nex-N2-mini, produced with llm-compressor and stored in the compressed-tensors format. The MoE expert stack is quantized to NVFP4 (W4A4) and the attention / output projections to FP8 (W8A8), shrinking the checkpoint from 70.2 GB to 24.8 GB (~2.8×) so the model fits comfortably on a single GPU.

---

Quantization Details

The model is quantized with compressed-tensors (quant_method: compressed-tensors, format: mixed-precision). The full recipe ships in recipe.yaml.

NVFP4 — W4A4 (nvfp4-pack-quantized)

Applied to the MoE feed-forward stack in layers 0–31 — this is the bulk of the parameters:

• model.language_model.layers.{0..31}.mlp.experts.{0..255}.{gate,up,down}_proj
• model.language_model.layers.{0..31}.mlp.shared_expert.{gate,up,down}_proj

FP8 — W8A8 (float-quantized)

Applied to attention, the linear-attention projections, the LM head, and the MoE stack of the last 8 layers (32–39), which are the most quantization-sensitive:

• self_attn.{q,k,v,o}_proj
• linear_attn.{in_proj_qkv,in_proj_z,out_proj}
• lm_head
• layers.{32..39}.mlp.experts.*.{gate,up,down}_proj and layers.{32..39}.mlp.shared_expert.{gate,up,down}_proj

Kept in BF16

• The full vision tower (model.visual.*, 333 tensors)
• MoE routers — mlp.gate, mlp.shared_expert_gate
• Linear-attention gating / state projections — linear_attn.in_proj_a, linear_attn.in_proj_b, linear_attn.conv1d, A_log, dt_bias, and all linear_attn.norm*
• Token embeddings and all RMSNorm weights

The KV cache is not quantized (kv_cache_scheme: null).

---

Hardware

NVFP4 reaches its full speedup on NVIDIA Blackwell GPUs (SM100 / SM120 — B200, GB200, RTX PRO 6000 Blackwell), which have native FP4 tensor cores. vLLM can still load and run this checkpoint on pre-Blackwell hardware through its NVFP4 dequantization/emulation path, in which case you get the memory savings but not the throughput gain.

At 24.8 GB of weights the model serves on a single 80 GB card with room for a long-context KV cache; the launch command below is tuned for that shape.

---

Usage

Deployment with vLLM

Note: Qwen3.5-MoE support, the NVFP4 compressed-tensors kernels, and the --language-model-only flag all require a recent vLLM build. Use a current release or nightly.

vllm serve /models/Nex-N2-Mini-NVFP4 \
  --served-model-name nex-n2 \
  --host 0.0.0.0 --port 8000 \
  --trust-remote-code --language-model-only \
  --tensor-parallel-size 1 \
  --max-model-len 131072 \
  --max-num-seqs 32 \
  --gpu-memory-utilization 0.60 \
  --max-num-batched-tokens 8192 \
  --enable-chunked-prefill

Flag notes:

• --language-model-only skips the (unquantized, BF16) vision tower and serves the model as a pure text LLM. Drop it if you want the multimodal path.
• --max-model-len 131072 is a conservative cap; the checkpoint's max_position_embeddings is 262144.
• --gpu-memory-utilization 0.60 and --max-num-seqs 32 leave generous headroom — raise them if you have spare VRAM and want more concurrency.
• --tensor-parallel-size 1 is sufficient at this precision. Increase it for extra KV-cache capacity, not for weight capacity.

Query it through the OpenAI-compatible API:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nex-n2",
    "messages": [{"role": "user", "content": "Write a Python function that merges overlapping intervals."}],
    "temperature": 0.7,
    "top_p": 0.95,
    "top_k": 40
  }'

Function Calling and Reasoning Parser

Nex-series models emit explicit reasoning traces and support robust function calling. To have vLLM split the reasoning trace from the final answer and parse tool calls into the OpenAI tool_calls schema, add:

  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder

Without these flags the reasoning trace and the <tool_call> blocks are returned inline in content, and you parse them yourself.

Recommended Sampling Parameters

• temperature: 0.7
• top_p: 0.95
• top_k: 40

Deployment with SGLang

The base model is best served with the customized sglang fork described in the Nex-N2-mini card. That fork does not currently consume compressed-tensors NVFP4 checkpoints — use the BF16 base model there, and this repository with vLLM.

---

Performance

The scores below are the published results for the BF16 base models and are reproduced here for reference. They were not re-measured on this NVFP4 checkpoint; expect a small degradation typical of 4-bit post-training quantization. If you benchmark it, we would welcome the numbers in the Community tab.

---

About Nex-N2

Nex-N2 is an agent model built for real-world productivity scenarios. With first-tier coding and agentic capabilities, it keeps driving complex, long-horizon tasks forward in real environments to deliver stable, end-to-end results.

Rather than treating reasoning, tool use, and environment execution as separate capabilities, Nex-N2 unifies them through an Agentic Thinking framework that connects requirement understanding, task planning, code implementation, environmental feedback, evaluation and debugging, and continuous iteration into a single closed loop. The framework has two parts:

• Adaptive Thinking lets the model decide on its own when to think and how deeply — executing simple actions quickly while reasoning thoroughly on critical decisions.
• Coherent Thinking carries one consistent reasoning paradigm across general reasoning and diverse agentic tasks, staying consistent across tasks and modalities to enable stable capability transfer.

Nex-N2 ships in two variants, both post-trained on the Qwen3.5 series: Nex-N2-Pro (built on Qwen3.5-397B-A17B) and Nex-N2-mini (built on Qwen3.5-35B-A3B-Base).

Model Architecture

Related Models

• Nex-N2-Pro: Hugging Face | ModelScope
• Nex-N2-mini (BF16 base): Hugging Face | ModelScope

License

Released under the Apache 2.0 license, matching the base model.