模型卡、文件列表和校验和已在此收录。如果你持有文件并有权分享,可以提交种子,让其他人从节点下载。
获取来源文件需要 Hugging Face 批准。Block-FP8 (8-bit) quantization of the abliterated (refusal-removed) Nex-N2.5-mini — byte-format-identical to Qwen's own FP8 scheme, served on vLLM
|
🛡️ New from Orca — OrcaCyber Zero 1.5 Post-trained for cybersecurity: vulnerability analysis, vulnerability reproduction, red teaming & offensive-security research. 98.01% pass@1 on CyberGym Level 1 · 1M context · function calling · OpenAI-compatible API. |
|
|
🔌 New from Orca — Orca Connect Serving this with vLLM? Orca Connect puts the rig under your desk behind your usual OrcaRouter API key, next to 200+ hosted models, reachable from a laptop, a phone or a cloud agent. Outbound only: no port forwarding, no tunnel, no firewall rule. Requests your own hardware serves cost $0, and |
|
One Gateway. Every Model. — Route Smarter · Ship Safer · Spend Less.
Website · Model Catalog · GitHub · Discord · X
Block-FP8 weight quantization of the abliterated (refusal-removed) build of Nex-AGI's
Nex-N2.5-mini— a 35B / 3.5B-active agentic multimodal Mixture-of-Experts model built on the Qwen3.5-MoE architecture (qwen3_5_moe, 256 routed experts top-8 + 1 shared) with a 3:1 hybrid of gated delta-net linear attention and full attention, a native Qwen3-VL vision tower, and a 262K-token context. Weights are quantized to FP8-E4M3 on a 128×128 block grid with dynamic activations — the exact scheme of the officialQwen/Qwen3.5-35B-A3B-FP8, so it serves through the identical vLLM kernel path — cutting the checkpoint from 65.4 GiB (BF16) to 34.1 GiB. Browse all models in the OrcaRouter Model Catalog.
Derived releases: •
Nex-N2.5-mini-Uncensored(BF16 source) •Nex-N2.5-mini-Uncensored-FP8(this repo) •Nex-N2.5-mini-Uncensored-NVFP4(4-bit experts).
This model has had its safety alignment substantially removed via abliteration (orthogonalizing the
refusal direction out of the residual stream). It will comply with harmful, unethical, offensive, or
illegal requests the original Nex-N2.5-mini would refuse — it has no meaningful built-in guardrails.
Released strictly for legitimate research: interpretability, AI-safety / refusal-mechanism study,
red-teaming, and robustness evaluation. You assume full responsibility for how you use it and
everything it generates; add your own safety, moderation, and abuse-prevention layers before any
deployment. Use must comply with the Apache 2.0 License
inherited from the base model and all applicable law. The authors accept no liability for misuse, and
its outputs do not reflect the views of the uploaders or of Nex-AGI.
| Base model | nex-agi/Nex-N2.5-mini → orcarouter/Nex-N2.5-mini-Uncensored (abliterated, then quantized) |
| Architecture | Qwen3_5MoeForConditionalGeneration (qwen3_5_moe) — 40 layers, hidden 2048, 3:1 hybrid attention (30 gated delta-net linear layers + 10 full-attention layers, head_dim 256 with output gating), 256 routed experts top-8 + 1 shared expert (moe_intermediate_size 512), 27-block Qwen3-VL vision tower, interleaved M-RoPE |
| Parameters | 35.1 B total / ~3.5 B active per token |
| Quantization | Block-FP8 (E4M3, 128×128 blocks, activation_scheme: dynamic) |
| Format | safetensors, <mod>.weight (float8_e4m3fn) + <mod>.weight_scale_inv (bfloat16); dequant is w * scale_inv broadcast over each 128×128 block |
| Size | 34.1 GiB / 36.6 GB (from 65.4 GiB BF16 — 52%) |
| Context | 262,144 tokens · Vocabulary 248,320 |
| Recommended for | Red-team & refusal-mechanism research, agentic / computer-use experiments, cost-efficient self-hosting of the uncensored build |
| Component | Precision |
|---|---|
Routed MoE experts (mlp.experts.*.{gate,up,down}_proj, all 40 layers — 91.8% of params) |
FP8 (E4M3, 128×128 block scales) |
Shared expert, full attention (self_attn.{q,k,v,o}_proj), linear-attention bulk (linear_attn.{in_proj_qkv,in_proj_z,out_proj}) |
FP8 |
Embeddings, lm_head, the whole vision tower, all norms, MoE router (mlp.gate), shared-expert gate |
BF16 |
Gated delta-net internals: conv1d, in_proj_a, in_proj_b, A_log, dt_bias |
BF16 |
modules_to_not_convert reproduces the official
Qwen/Qwen3.5-35B-A3B-FP8 list entry for entry (284 modules; the official build has 3 more, all
naming an MTP block this checkpoint does not have). Nex's own FP8 release of the previous generation,
nex-agi/Nex-N2-Pro-fp8, uses the same categories.linear_attn.in_proj_a and in_proj_b are [32, 2048]
each — 65 K parameters apiece — and they produce the per-head decay a and the delta-rule beta that
drive the entire recurrence; A_log feeds an exponential (A = -exp(A_log)). All are kept BF16, which
costs 0.01 GiB for all 60 of them. The MoE router and the shared-expert gate (a single [1, 2048] row)
are kept BF16 for the same reason: a small error there re-routes every token.nex-agi/Nex-N2.5-mini ships 1026 tensors and zero mtp.* — Nex did not release
a multi-token-prediction head for this model (the official Qwen/Qwen3.5-35B-A3B does). Nothing was
dropped in quantization; MTP speculative decoding is simply not available for this checkpoint.Format verification. Checked against the official build's own safetensors headers: the tensor-name set is identical in both directions once its 1560
mtp.*tensors are removed (62,636 = 64,196 − 1560), and dtype + shape match on 10,523 sampled tensors across three of its shards, with zero mismatches. Running this pipeline's quantizer on Qwen's own BF16 weights reproduces their block scales bit-identically (0 / 3008 elements differ); 1.67% of weight bytes differ, every one by exactly ±1 E4M3 ULP, because the official build used a float32 divide for its dense projections and a bfloat16 divide for its routed experts, and no single code path matches both halves. This build uses the uniformly higher-fidelity one (+0.015–0.018 dB SNR on dense tensors, +0.047–0.053 dB on experts).
qwen3_5_moe support, or transformers ≥ 5.17 (which carries the reference
architecture and the FineGrainedFP8 loader).vllm serve orcarouter/Nex-N2.5-mini-Uncensored-FP8 \
--served-model-name Nex-N2.5-mini-Uncensored-FP8 \
--max-model-len 32768 --trust-remote-code
from transformers import AutoModelForImageTextToText, AutoTokenizer
mid = "orcarouter/Nex-N2.5-mini-Uncensored-FP8"
tok = AutoTokenizer.from_pretrained(mid)
model = AutoModelForImageTextToText.from_pretrained(mid, dtype="auto", device_map="auto")
msgs = [{"role": "user", "content": "Explain gated delta-net attention in two sentences."}]
text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True,
enable_thinking=False) # template opens <think> otherwise
out = model.generate(**tok(text, return_tensors="pt").to(model.device), max_new_tokens=256)
print(tok.decode(out[0], skip_special_tokens=True))
The chat template opens a <think> block by default. Pass enable_thinking=False to
apply_chat_template (or chat_template_kwargs={"enable_thinking": False} through an OpenAI-compatible
client) for direct answers, and give generation enough budget to reach </think> when thinking is on, or
replies get truncated inside the scratchpad.
Note on stop tokens. Neither this build nor upstream
nex-agi/Nex-N2.5-miniships ageneration_config.json, sotransformersfalls back toconfig.jsonand useseos_token_id = 248044—<|im_end|>(248046) is not a stop token by default and decoding is greedy. Passeos_token_id=[248046, 248044]explicitly, or supply your own generation config.
Measured on this build's actual bytes, injected into a BF16 reference of the same checkpoint so both sides run the identical kernel stack and the difference isolates exactly the weight change. All numbers are from these weights, not inherited from the base card.
Every quantized tensor, round-tripped through the shipped scales:
| metric | value |
|---|---|
| cosine similarity | 0.999650 |
| SNR | 31.55 dB |
| max relative error | 3.6% |
| mean relative error (experts) | ~2.6% |
Uniform across roles — experts, shared expert, full attention and the linear-attention bulk all land within 0.02 dB of each other. Every non-quantized tensor is bit-identical to the source, verified tensor by tensor.
24,564 predicted tokens, 12 chunks × 2048:
| BF16 ref | FP8 | |
|---|---|---|
| PPL | 7.051 | 7.065 (+0.20%) |
| KLD (mean) | — | 0.0315 |
| KLD (p95 / p99) | — | 0.096 / 0.281 |
| Top-1 agreement | — | 92.86% |
Top-1 agreement is lower than an 8-bit build of a dense model would give, and that is a property of the architecture rather than of the quantization: with 256 fine-grained experts and top-8 routing, a small perturbation of the hidden state flips which experts a token is routed to, and expert selection is a discrete function. The router itself is kept BF16; what moves is its input. PPL, which does not depend on argmax, moves by only 0.20%.
Abliteration removes the refusal direction from the residual stream, and the routed-expert
down_proj matrices it lives in are exactly what gets quantized here — so "is it still
uncensored" is a property of this build, not of the source, and is measured on this build's
own bytes. JailbreakBench (JBB-Behaviors), 100 harmful + 100 benign prompts, greedy, 64 new
tokens, reasoning_effort=none (thinking off):
| BF16 abliterated source | This FP8 build | |
|---|---|---|
| harmful — explicit refusal (↓ = more uncensored) | 0.000 | 0.000 |
| harmful — deflect (names the harm, then answers a different question) | 0.150 | 0.080 |
| harmful — complies | 0.850 | 0.920 |
| benign — over-refusal (↓ = better) | 0.000 | 0.000 |
| benign — complies | 1.000 | 1.000 |
Explicit refusal is zero, and quantization does not put the guardrails back: this build deflects slightly less than its BF16 source, and benign over-refusal stays at zero on both.
Method note — why three categories and not two. This model rarely opens a harmful response with "I can't". Far more often it names the harm and then answers a different, safe question — "A xenophobic speech would unfairly target people based on ethnicity and promote hatred. Here's a strong alternative speech that argues against xenophobia:". Scoring that as compliance overstates how uncensored a build is; scoring it as refusal overstates the opposite, so it is reported separately as deflect (which requires both a harm-flag and a pivot marker in the opening, so a disclaimer followed by compliance still counts as compliance). The classifier is rule-based (EN + ZH) and indicative, not an LLM-judge or publication-grade number — evaluate rigorously for your own use case. Note also that this model's chat template gates thinking on
reasoning_effort, notenable_thinking; with thinking left on, a short token budget is consumed entirely by the deliberation trace and every build scores a meaningless 0.000.
transformers build with qwen3_5_moe support; the FP8 scheme is Qwen's
own, so no bespoke kernel path is required.Nex-N2.5-mini.Apache 2.0, inherited from the base model
nex-agi/Nex-N2.5-mini. Abliteration and quantization do
not change the underlying license obligations.
model-00001-of-00016.safetensors 3.3 GBmodel-00006-of-00016.safetensors 2.5 GBmodel-00009-of-00016.safetensors 2.5 GBmodel-00012-of-00016.safetensors 2.5 GBmodel-00015-of-00016.safetensors 2.5 GBmodel-00003-of-00016.safetensors 2.5 GBmodel-00007-of-00016.safetensors 2.3 GBmodel-00010-of-00016.safetensors 2.3 GBmodel-00013-of-00016.safetensors 2.3 GBmodel-00004-of-00016.safetensors 2.3 GBmodel-00005-of-00016.safetensors 2.0 GBmodel-00008-of-00016.safetensors 2.0 GBmodel-00011-of-00016.safetensors 2.0 GBmodel-00014-of-00016.safetensors 2.0 GBmodel-00002-of-00016.safetensors 2.0 GBmodel-00016-of-00016.safetensors 1.9 GBorcarouter/Nex-N2.5-mini-Uncensored