Qwen2Moe & GGUF EditionMeditor-2-3.4B-Universal flattens the hierarchical routing graph of Yo-Purple/Meditor-2-3.4B-A1.8B-HMoE into standard Hugging Face Qwen2MoeForCausalLM (qwen2_moe) and ready-to-run GGUF quantizations (BF16 through Q2_K / IQ4_XS).
| Generation & Variant | Hugging Face Repo | Base Backbone | Params (Total / Active) | UniBench-100 Mean (/10) | Pass Rate | VRAM Footprint |
|---|---|---|---|---|---|---|
| Meditor 1 (Native MoE) | Yo-Purple/Meditor-1-2.2B-A500M-MoE |
Qwen2.5-0.5B | 2.22B / 0.57B | 2.09 (37/100) · 2.42 (Best M1) | 1.0% (Best M1) | 4.16 GB (BF16) |
| Meditor 1 (Universal) | Yo-Purple/Meditor-1-500M-Universal |
Qwen2.5-0.5B (qwen2) |
0.49B / 0.49B | 2.42 (100/100 BF16) | 1.0% | 0.93 GB / GGUF |
| Meditor 2 (Native HMoE) | Yo-Purple/Meditor-2-3.4B-A1.8B-HMoE |
SmolLM2-1.7B | 3.41B / 1.83B | 3.28 (37/100) · 4.12 (Best M2) | 22.0% (Best M2) | 6.42 GB (BF16) |
| Meditor 2 (Universal) (This Model) | Yo-Purple/Meditor-2-3.4B-Universal |
SmolLM2-1.7B (qwen2_moe) |
3.41B / 1.83B | 4.12 (100/100 BF16) · 4.00 (Q4_K_M) | 22.0% / 18.0% | 6.38 GB / GGUF |
| Meditor 3 (Native MoE) | Yo-Purple/Meditor-3-4B-MoE |
Qwen3-4B (meditor3) |
4.16B / 4.04B–4.07B | 7.31 (100/100, Rank #1/36) | 69.0% (7.0% Uniq) | 5.43 GiB peak (bf16_low_vram) |
| Meditor 3 (Universal GGUF) | Yo-Purple/Meditor-3-4B-Universal |
Qwen3-4B (k2-horizon) |
4.16B (BF16) · Q8_0 · Q4_K_M |
Compiled GGUF from M3 (7.31 native) |
LM Studio / llama.cpp | 4.37–5.95 GiB (quantized) |
UniBench-100) Cross-Generation ComparisonAll benchmark comparisons below come directly from the unified UniBench-100 evaluation suite (100 open-ended synthesis questions across 10 cognitive domains, graded out of 10.0 with pass threshold >= 6.0 by gemini-3.8-flash).
| Rank (of 36 Runs) | Model Run | Series Generation | Evaluated | Mean Score (/10) | Pass Rate (>=6.0) | Unique Solve Rate | Eval Speed |
|---|---|---|---|---|---|---|---|
| #1 | Meditor-3-4B-MoE (Released) | Meditor 3 (Native) | 100 / 100 | 7.31 | 69.0% | 7.0% | 35.7 tok/s |
| #2 | Meditor-3 Prior Uncontested (BF16) | Meditor 3 (Prior R&D) | 100 / 100 | 4.62 | 30.0% | 1.0% | 35.7 tok/s |
| #5 | Meditor 3 Beta (BF16) | Meditor 3 (Prior R&D) | 100 / 100 | 4.31 | 22.0% | 1.0% | 19.0 tok/s |
| #11 | Meditor 2 Universal (BF16) | Meditor 2 Best | 100 / 100 | 4.12 | 22.0% | 0.0% | 27.5 tok/s |
| #18 | Meditor 2 Universal (Q4_K_M) | Meditor 2 Quantized | 100 / 100 | 4.00 | 18.0% | 0.0% | 141.2 tok/s |
| #29 | Meditor 2 (Native HMoE) | Meditor 2 Native | 37 / 100 | 3.28 | 8.1% | 0.0% | 48.7 tok/s |
| #30 | Meditor 1 Universal (BF16) | Meditor 1 Best | 100 / 100 | 2.42 | 1.0% | 0.0% | 163.4 tok/s |
| #31 | Meditor 1 Universal (Quant) | Meditor 1 Quantized | 100 / 100 | 2.35 | 0.0% | 0.0% | 171.3 tok/s |
| #33 | Meditor 1 (Native MoE) | Meditor 1 Native | 37 / 100 | 2.09 | 2.7% | 0.0% | 141.9 tok/s |
Key Takeaway: Meditor-3-4B-MoE delivers +3.19 points (
+77.4%relative score jump) over the best Meditor 2 run (4.12), +4.89 points (3.02xscore) over the best Meditor 1 run (2.42), and +2.69 points over the strongest earlier Meditor 3 research run (4.62).
| UniBench-100 Domain | Meditor 1 Best (2.42) |
Meditor 2 Best (4.12) |
Meditor 3 Released (7.31) |
Gain vs M2 | M3 Pass Rate |
|---|---|---|---|---|---|
| Critical Thinking & Argument Evaluation | 3.51 | 5.98 | 9.09 | +3.11 |
100.0% |
| Creative & Stylistic Expression | 2.91 | 4.95 | 8.38 | +3.43 |
90.0% |
| Humanities, Philosophy & Ethics | 2.30 | 4.91 | 8.35 | +3.44 |
80.0% |
| Physical & Life Sciences | 1.68 | 3.38 | 7.41 | +4.03 |
80.0% |
| Mathematics & Quantitative Estimation | 2.36 | 1.99 | 7.27 | +5.28 |
60.0% |
| Factuality & Hallucination Resistance | 1.32 | 5.56 | 7.09 | +1.53 |
60.0% |
| Real-World Tactical Strategy & Decision-Making | 2.10 | 3.72 | 6.98 | +3.26 |
60.0% |
| Instruction Following & Strict Constraints | 3.65 | 4.62 | 6.82 | +2.20 |
50.0% |
| Coding & Software Engineering | 2.15 | 3.26 | 6.46 | +3.20 |
70.0% |
| Logic & Abstract Reasoning | 2.19 | 2.82 | 5.27 | +2.45 |
40.0% |
Meditor-2-3.4B-Universal)| Architectural Parameter | Value |
|---|---|
| Model Class | Qwen2MoeForCausalLM (qwen2_moe) |
| Backbone Specs | 24 layers, d_model=2048, 32Q / 32KV heads |
| FFN / MoE Layout | 1 Shared Expert (8192) + 15 Routed Experts (768, top-1 per token) |
| Parameters | 3,411,003,392 Total · 1,825,507,328 Active (53.52%) |
| Context Window | 524,288 tokens (rope_theta=130,000.0) |
| Evaluation Benchmark | Observed Result |
|---|---|
UniBench-100 (BF16) |
4.12 / 10 (22.0% pass, 100/100 evaluated @ 27.5 tok/s — Rank #11 / 36) |
UniBench-100 (Q4_K_M) |
4.00 / 10 (18.0% pass, 100/100 evaluated @ 141.2 tok/s — Rank #18 / 36) |
| WikiText-2 Perplexity | 2.48 (loss 0.9063) |
ARC Subset (12 Questions) |
100.0% (12 / 12) |
Arithmetic (10 Questions) |
60.0% raw · 100.0% with tool assist |
PyTorch Speed & VRAM (seq_256) |
55.51 tok/s · 6.38 GB BF16 VRAM |
qwen2_moe): model.safetensors, config.json, tokenizer filesmeditor-2-3.4b-universal-bf16.gguf, meditor-2-3.4b-universal-q8_0.gguf, meditor-2-3.4b-universal-q6_k.gguf, meditor-2-3.4b-universal-q5_k_m.gguf, meditor-2-3.4b-universal-q5_k_s.gguf, meditor-2-3.4b-universal-q4_k_m.gguf, meditor-2-3.4b-universal-q4_k_s.gguf, meditor-2-3.4b-universal-q3_k_m.ggufmradermacher): Meditor-2-3.4B-Universal.IQ4_XS.gguf, Meditor-2-3.4B-Universal.Q3_K_L.gguf, Meditor-2-3.4B-Universal.Q3_K_S.gguf, Meditor-2-3.4B-Universal.Q2_K.ggufimport torch
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "Yo-Purple/Meditor-2-3.4B-Universal"
tok = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(repo_id, torch_dtype=torch.bfloat16, device_map="auto")
inputs = tok("Summarize Newton's second law:", return_tensors="pt").to(model.device)
print(tok.decode(model.generate(**inputs, max_new_tokens=96)[0], skip_special_tokens=True))
Or run via llama.cpp:
llama-server -m meditor-2-3.4b-universal-q4_k_m.gguf -ngl 99 -c 8192
Released under Apache 2.0. Built on HuggingFaceTB/SmolLM2-1.7B-Instruct.
HuggingFaceTB/SmolLM2-1.7B-Instruct