模型卡、文件列表和校验和已在此收录。如果你持有文件并有权分享,可以提交种子,让其他人从节点下载。
获取来源文件需要 Hugging Face 批准。MLX quants (2-bit → 8-bit) of the abliterated (refusal-removed) Nex-N2.5-mini — for Apple Silicon
|
🛡️ New from Orca — OrcaCyber Zero 1.5 Post-trained for cybersecurity: vulnerability analysis, vulnerability reproduction, red teaming & offensive-security research. 98.01% pass@1 on CyberGym Level 1 · 1M context · function calling · OpenAI-compatible API. |
|
|
🔌 New from Orca — Orca Connect Have a rig running Ollama, LM Studio, vLLM, llama.cpp or SGLang? Orca Connect puts it behind your usual OrcaRouter API key, next to 200+ hosted models, reachable from a laptop, a phone or a cloud agent. Outbound only: no port forwarding, no tunnel, no firewall rule. Requests your own hardware serves cost $0, and |
|
One Gateway. Every Model. — Route Smarter · Ship Safer · Spend Less.
Website · Model Catalog · GitHub · Discord · X
| Quant | Location | Size | ΔPPL vs BF16 |
|---|---|---|---|
| 8-bit | 8bit/ |
38.70 GB | +0.7 % |
| 6-bit | 6bit/ |
30.65 GB | +0.5 % |
| 4-bit (recommended) | 4bit/ |
19.85 GB | +2.8 % |
| 3-bit | 3bit/ |
18.17 GB | +2.9 % |
| 2-bit | 2bit/ |
14.15 GB | +11.0 % |
Each tier is a complete model in its own subfolder. These are dynamic (mixed-precision) MLX quants: routed experts carry the base bits, attention is pinned to 8-bit in every tier, and the router / delta-net decay and beta projections / norms stay BF16 — so the footprint is above a uniform quant, by design (see below). 6-bit and 8-bit are indistinguishable here, so 6-bit is the best fidelity per byte; the 4-bit tier is built to a hard 20 GB ceiling and fits a 24 GB machine.
An abliterated MLX build of
nex-agi/Nex-N2.5-mini— a 35B / 3.5B-active agentic multimodal Mixture-of-Experts model on the Qwen3.5-MoE architecture (qwen3_5_moe, 256 routed experts top-8 + 1 shared) with a 3:1 hybrid of gated delta-net linear attention and full attention, a native Qwen3-VL vision tower, and a 262K-token context — quantized to MLX for Apple Silicon and the MLX CUDA backend. Runs undermlx-vlm >= 0.7.0, which hasqwen3_5_moesupport in mainline.Derived releases: •
Nex-N2.5-mini-Uncensored(BF16 source) •…-FP8(block-FP8, mirrors Qwen's scheme) •…-NVFP4(experts-only NVFP4, for vLLM) •…-GGUF(llama.cpp) •…-MLX(this repo).
This model has had its safety alignment substantially removed via abliteration (orthogonalizing the refusal direction out of the residual stream). As a direct consequence:
Nex-N2.5-mini would refuse. It has no meaningful built-in guardrails.By downloading or using this model you acknowledge and accept the above.
Models are only half the system.
OrcaCode Review turns every model listed on OrcaRouter into a production code-review agent:
Open model. Open harness. Open bill.
| Base model | nex-agi/Nex-N2.5-mini |
| Architecture | Qwen3_5MoeForConditionalGeneration (qwen3_5_moe) — 40 layers, hidden 2048, 3:1 hybrid attention (30 gated delta-net linear layers + 10 full-attention, head_dim 256 with output gating), 256 routed experts top-8 + 1 shared expert (moe_intermediate_size 512), 27-block Qwen3-VL vision tower, interleaved M-RoPE |
| Parameters | 35.1 B total / ~3.5 B active per token |
| Modification | Abliteration (refusal-direction removal) then MLX quantization |
| Quantization | MLX affine, dynamic mixed-precision (recipe below); attention pinned 8-bit |
| Format | safetensors (MLX), 16 shards per tier |
| Context | 262,144 tokens · Vocabulary 248,320 |
Calibration-free, architecture-aware mixed precision: tensor-role priors spend bits where they
matter instead of running a per-layer search. MLX affine costs bits + 32/group_size bits per weight
— a bf16 scale and bias per group — so group size is as strong a size knob as the bit-width
(4-bit is 4.50 bpw at gs64, 4.25 at gs128), and both are part of the recipe.
| Component | Params | 2-bit | 3-bit | 4-bit | 6-bit | 8-bit | Policy |
|---|---|---|---|---|---|---|---|
Routed experts gate_proj / up_proj |
21.5 B | 2 gs64 |
3 gs64 |
4 gs128 |
6 gs64 |
8 gs64 |
base |
Routed experts down_proj |
10.7 B | 3 gs64 |
4 gs64 |
4 gs128 |
6 gs64 |
8 gs64 |
base +1 at low tiers |
| Full attention + gated delta-net bulk | 1.28 B | 8 | 8 | 8 | 8 | 8 | pinned 8-bit |
lm_head |
0.51 B | 6 | 6 | 6 | BF16 | BF16 | held above embeddings |
embed_tokens |
0.51 B | 4 | 4 | 4 | BF16 | BF16 | lookup table |
| Vision tower | 0.45 B | 6 | 6 | 6 | BF16 | BF16 | linear_fc2 always BF16 |
| Shared expert | 0.13 B | 6 | 6 | 6 | 8 | 8 | |
Router, shared_expert_gate, in_proj_a/b, A_log, dt_bias, conv1d, all norms |
0.03 B | BF16 | BF16 | BF16 | BF16 | BF16 | never quantized |
lm_head is held above embed_tokens — an embedding is a lookup table; lm_head produces the
logits. A mixed FP8+NVFP4 experiment on this checkpoint measured that taking lm_head down cost
more KLD than protecting eight whole layers of experts gained.in_proj_a / in_proj_b are [32, 2048] each — 65 K
parameters apiece — and produce the per-head decay a and the delta-rule beta that drive the whole
recurrence; A_log feeds an exponential. The router and the shared-expert gate (a single [1, 2048]
row) steer every token. All stay BF16, for 0.03 GB.mlp.linear_fc2 has in_features = 4304, which none of MLX's group sizes divide
(4304 = 16 × 269), so those 27 tensors ship BF16 rather than being forced onto a size they do not fit.{group_size, bits} overrides are written to each tier's config.json → quantization.Refusal-direction removal following Arditi et al. (2024), Refusal in Language Models Is Mediated by
a Single Direction — a single direction orthogonalized out of every residual-writing matrix and baked
into the BF16 checkpoint these builds were quantized from. On this architecture the direction lives
almost entirely in the routed-expert down_proj matrices — the very weights these tiers quantize — so
"is it still uncensored" is a property of each build, and was measured on each build's own bytes:
JailbreakBench, 100 harmful + 100 benign prompts, greedy, reasoning_effort=none. Explicit refusal
is 0.000 at every tier, down to 2-bit, and benign over-refusal is 0.000 — quantizing the routed experts
does not put the guardrails back. Refusal is reduced, not removed: where the model resists it names
the harm and answers a different, safe question rather than declining outright (0.07–0.14 across tiers
against 0.15 for the BF16 source — sampling noise, not a trend). See the base
…-FP8 card for the full method and
scoring caveats.
Each build's actual quantized bytes against the abliterated BF16 source these were quantized from, on wikitext-2 test, 12 chunks × 2048 tokens (24,564 predicted tokens). Lower KLD and higher Top-1 = closer to the source.
| Build | Size | PPL | ΔPPL | Mean KLD | KLD p95 | Top-1 agreement |
|---|---|---|---|---|---|---|
| BF16 uncensored (ref) | 65.4 GB | 7.051 | — | — | — | 100 % |
| 8-bit | 38.70 GB | 7.097 | +0.7 % | 0.029 | 0.083 | 93.3 % |
| 6-bit | 30.65 GB | 7.087 | +0.5 % | 0.029 | 0.089 | 93.3 % |
| 4-bit | 19.85 GB | 7.250 | +2.8 % | 0.063 | 0.210 | 89.6 % |
| 3-bit | 18.17 GB | 7.258 | +2.9 % | 0.097 | 0.333 | 87.1 % |
| 2-bit | 14.15 GB | 7.826 | +11.0 % | 0.258 | 0.957 | 79.5 % |
All three signals agree on the ordering: 6-bit is lossless and 8-bit buys nothing over it, 4-bit is the sweet spot, 3-bit is barely behind at 8 % less size, and 2-bit costs a lot — pick it for fit, not quality. Two caveats: the 8-bit KLD of 0.029 is the measurement floor, not its quantization error, so 4-bit's real distance from BF16 is nearer 0.034 than 0.063; and Top-1 is low for these bit-widths because of the architecture, not the quantizer — with 256 fine-grained experts top-8, a small perturbation flips which experts a token routes to, and that is a discrete function (PPL, which ignores argmax, moves 0.7 % at 8-bit).
The 4-bit tier was measured twice by independent paths — on mlx-vlm, and by unpacking its bit-stream
in PyTorch and injecting into a BF16 transformers model — which agree (PPL 7.250 vs 7.278, KLD 0.0629
vs 0.0619, Top-1 89.63 % vs 89.66 %). Per-tensor weight-space fidelity is recorded at pack time in each
tier's fidelity_summary.json (4-bit routed experts: cosine 0.9948, SNR 19.9 dB).
pip install -U "mlx-vlm>=0.7.0"
# pick a tier by subfolder
python -m mlx_vlm.generate --model orcarouter/Nex-N2.5-mini-Uncensored-MLX/4bit \
--prompt "Explain gated delta-net attention in two sentences." --max-tokens 512 --temp 0
# vision
python -m mlx_vlm.generate --model orcarouter/Nex-N2.5-mini-Uncensored-MLX/4bit \
--image screenshot.png --prompt "What is on screen?" --max-tokens 512
Nex-N2.5-mini is a vision-language model, so it runs under mlx-vlm (not mlx-lm). Its chat
template gates the <think> block on reasoning_effort, not enable_thinking — pass
reasoning_effort="none" for direct answers, and otherwise give generation enough --max-tokens to
reach </think> or the reply is truncated inside the scratchpad.
Stop tokens. Neither this build nor upstream
nex-agi/Nex-N2.5-miniships ageneration_config.json, so a loader falling back toconfig.jsonuseseos_token_id = 248044—<|im_end|>(248046) is not a stop token by default.
Only 8 of 256 routed experts are active per token, so decode is far faster than 35 B suggests, but the full weights must fit in unified memory: budget the tier's size plus KV cache — 4-bit on a 24 GB Mac, 6-bit on 36 GB, 8-bit on 48 GB.
Nex-N2.5-mini.mtp.* tensors, so speculative decoding via MTP is unavailable.
Nothing was dropped in quantization.Apache 2.0, inherited from nex-agi/Nex-N2.5-mini.
Abliteration and quantization do not change the underlying license obligations.
8bit/model-00001-of-00016.safetensors 3.4 GB6bit/model-00001-of-00016.safetensors 3.1 GB8bit/model-00009-of-00016.safetensors 2.6 GB8bit/model-00012-of-00016.safetensors 2.6 GB8bit/model-00015-of-00016.safetensors 2.6 GB8bit/model-00006-of-00016.safetensors 2.6 GB8bit/model-00003-of-00016.safetensors 2.6 GB8bit/model-00010-of-00016.safetensors 2.4 GB8bit/model-00013-of-00016.safetensors 2.4 GB8bit/model-00007-of-00016.safetensors 2.4 GB8bit/model-00004-of-00016.safetensors 2.4 GB8bit/model-00005-of-00016.safetensors 2.1 GB8bit/model-00008-of-00016.safetensors 2.1 GB8bit/model-00011-of-00016.safetensors 2.1 GB8bit/model-00014-of-00016.safetensors 2.1 GB8bit/model-00002-of-00016.safetensors 2.1 GB6bit/model-00012-of-00016.safetensors 2.0 GB6bit/model-00015-of-00016.safetensors 2.0 GB6bit/model-00009-of-00016.safetensors 2.0 GB6bit/model-00006-of-00016.safetensors 2.0 GB6bit/model-00003-of-00016.safetensors 2.0 GB8bit/model-00016-of-00016.safetensors 1.9 GB6bit/model-00013-of-00016.safetensors 1.9 GB6bit/model-00010-of-00016.safetensors 1.9 GB6bit/model-00007-of-00016.safetensors 1.9 GB6bit/model-00004-of-00016.safetensors 1.9 GB6bit/model-00016-of-00016.safetensors 1.7 GB6bit/model-00005-of-00016.safetensors 1.6 GB6bit/model-00008-of-00016.safetensors 1.6 GB6bit/model-00014-of-00016.safetensors 1.6 GB6bit/model-00011-of-00016.safetensors 1.6 GB6bit/model-00002-of-00016.safetensors 1.6 GB4bit/model-00001-of-00016.safetensors 1.6 GB3bit/model-00001-of-00016.safetensors 1.5 GB4bit/model-00015-of-00016.safetensors 1.4 GB4bit/model-00012-of-00016.safetensors 1.4 GB4bit/model-00009-of-00016.safetensors 1.4 GB4bit/model-00006-of-00016.safetensors 1.4 GB4bit/model-00003-of-00016.safetensors 1.4 GB2bit/model-00001-of-00016.safetensors 1.3 GB4bit/model-00013-of-00016.safetensors 1.3 GB4bit/model-00010-of-00016.safetensors 1.3 GB4bit/model-00004-of-00016.safetensors 1.3 GB4bit/model-00007-of-00016.safetensors 1.3 GB3bit/model-00015-of-00016.safetensors 1.2 GB3bit/model-00012-of-00016.safetensors 1.2 GB3bit/model-00009-of-00016.safetensors 1.2 GB3bit/model-00006-of-00016.safetensors 1.2 GB3bit/model-00003-of-00016.safetensors 1.2 GB3bit/model-00010-of-00016.safetensors 1.1 GB3bit/model-00013-of-00016.safetensors 1.1 GB3bit/model-00007-of-00016.safetensors 1.1 GB3bit/model-00004-of-00016.safetensors 1.1 GB4bit/model-00005-of-00016.safetensors 1.1 GB4bit/model-00008-of-00016.safetensors 1.1 GB4bit/model-00014-of-00016.safetensors 1.1 GB4bit/model-00011-of-00016.safetensors 1.1 GB4bit/model-00002-of-00016.safetensors 1.1 GB3bit/model-00005-of-00016.safetensors 1.0 GB3bit/model-00008-of-00016.safetensors 1.0 GB3bit/model-00014-of-00016.safetensors 1.0 GB3bit/model-00011-of-00016.safetensors 1.0 GB3bit/model-00002-of-00016.safetensors 1.0 GB2bit/model-00015-of-00016.safetensors 935 MB2bit/model-00009-of-00016.safetensors 935 MB2bit/model-00012-of-00016.safetensors 935 MB2bit/model-00006-of-00016.safetensors 935 MB2bit/model-00003-of-00016.safetensors 935 MB4bit/model-00016-of-00016.safetensors 874 MB2bit/model-00010-of-00016.safetensors 850 MB2bit/model-00013-of-00016.safetensors 850 MB2bit/model-00007-of-00016.safetensors 850 MB2bit/model-00004-of-00016.safetensors 850 MB3bit/model-00016-of-00016.safetensors 832 MB2bit/model-00005-of-00016.safetensors 800 MB2bit/model-00008-of-00016.safetensors 800 MB2bit/model-00011-of-00016.safetensors 800 MB2bit/model-00014-of-00016.safetensors 800 MB2bit/model-00002-of-00016.safetensors 800 MB2bit/model-00016-of-00016.safetensors 731 MBorcarouter/Nex-N2.5-mini-Uncensored