Qwopus3.6-35B-A3B-v1 — GGUF Quantizations

GGUF quantizations of Jackrong/Qwopus3.6-35B-A3B-v1 for use with llama.cpp and compatible runtimes.

📎 Vision support: Place mmproj.gguf in the same directory as the main .gguf file to enable visual capabilities.

---

Available Files

---

About Qwopus3.6-35B-A3B-v1

Qwopus3.6-35B-A3B-v1 is a reasoning-enhanced MoE model fine-tuned on top of Qwen3.6-35B-A3B by Jackrong. It inherits the base model's hybrid sparse MoE architecture — combining Gated DeltaNet linear attention with standard gated attention layers — while sharpening reasoning depth through a structured three-stage supervised fine-tuning process.

Key specs:

• Total parameters: 35B
• Active parameters per token: 3B
• Experts: 256 routed experts, top-K routing
• Context window: 262k tokens
• Multimodal: Vision + tool calling support

⚠️ Community release notice: This model has not undergone complete performance evaluation or safety testing. Released as an experimental community version for research and exploration.

---

Training Overview

Fine-tuning used LoRA with ~9% of model parameters participating in updates. Training followed a three-stage curriculum:

Stage 1 — Format Establishment: Short-to-medium length, format-stable reasoning samples to establish structured output.

Stage 2 — Complexity Scaling & Multi-Teacher Distillation: Gradually increasing proportion of complex reasoning samples distilled from a 27B teacher model.

Stage 3 — Long-Context Reinforcement & Anti-Drift: Long-context reasoning with 10% short sample replay to prevent catastrophic forgetting.

---

Known Issues

• 9% trainable parameter ratio is a risky configuration for this MoE architecture
• Known incompatibilities between PEFT/LoRA, Transformers 5.x fused expert pattern, and Unsloth patches
• LoRA weight merging may fail with ModuleNotFoundError: Could not import module 'Qwen3_5MoeForConditionalGeneration'

These are upstream issues in the source model — the GGUF files in this repo are converted from the merged weights and are unaffected.

---

Usage with llama.cpp

Download

# Recommended — Q4_K_M
huggingface-cli download barozp/Qwopus3.6-35B-A3B-v1-GGUF \
    Qwopus3.6-35B-A3B-v1-Q4_K_M.gguf --local-dir ./

# With vision support
huggingface-cli download barozp/Qwopus3.6-35B-A3B-v1-GGUF \
    Qwopus3.6-35B-A3B-v1-Q4_K_M.gguf mmproj.gguf --local-dir ./

CLI

# Text only
llama-cli -m Qwopus3.6-35B-A3B-v1-Q4_K_M.gguf --jinja -ngl 99 -c 32768

# With vision
llama-cli -m Qwopus3.6-35B-A3B-v1-Q4_K_M.gguf --mmproj mmproj.gguf --jinja -ngl 99 -c 32768

# Server
llama-server -m Qwopus3.6-35B-A3B-v1-Q4_K_M.gguf --mmproj mmproj.gguf --jinja -ngl 99 -c 32768 --port 8080

Python

from llama_cpp import Llama

llm = Llama.from_pretrained(
    repo_id="barozp/Qwopus3.6-35B-A3B-v1-GGUF",
    filename="Qwopus3.6-35B-A3B-v1-Q4_K_M.gguf",
    n_gpu_layers=-1,
    n_ctx=32768,
)

response = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain the traveling salesman problem."},
    ]
)
print(response["choices"][0]["message"]["content"])

---

VRAM Requirements

For partial GPU offloading use -ngl <layers> to fit within available VRAM.

---

Quantization Details

• Source: Jackrong/Qwopus3.6-35B-A3B-v1 (BF16 safetensors)
• Converted with: llama.cpp convert_hf_to_gguf.py
• Quantized with: llama-quantize
• IQ quants: llama-quantize --imatrix with WikiText-2 calibration data
• Quantized by: barozp

---

License

Apache 2.0 — inherited from Qwen3.6-35B-A3B.