Loading…
Loading…
⚡ 40.1 tok/s on your CPU (i5-6200U 2C/4T) + 37 MB RAM — gguf2bin interface
Runtime: AnonymoDGH/gguf2bin on GitHub — C99 G2BX mmap + AVX2 + Vulkan
Qwen3-0.6B + 6 custom technologies — gguf2bin runtime (C99, AVX2, Vulkan)
Perfect r128 20k · 1.00 loss (3.52→1.00) · 62.7% SecEval · 520k lines 465MB (110 shards
rdru200m6.9GB +code_search_net20k)
Your CPU (i5-6200U 2C/4T DDR3L 9.4 GB/s) — measured bench -n32 min3 with --fast:
| Model | Weights (mmap) | Runtime RAM | decode | prefill | Your RAM knob |
|---|---|---|---|---|---|
| Qwen3-0.6B Q4 | 339 MB | 511 MB | 24.7 | 38.6 | --q8-kv → 3.8× less, -c 512 → 139 MB |
| +MV 0.5 | 339 MB | 511 MB | 40.1 (+62%) | 47.9 | --mv 0.5 |
| +BVH 15% | 339 MB | 511 MB | 24.8 (2.5× on ctx32k) | — | --bvh |
| +Particle-SousVide | 339 MB | 511 MB | 24.7 | 38.6 | train 5× faster |
| 2GB machine | 339 MB | 37 MB | 24.7 | — | --swap D:\kv.swap or --max-ram 2048 |
What counts on 2GB? Weights = mmap evictable (❌), KV cache = F32→Q8_0 (--q8-kv), buffers + tokenizer ~60 MB (✅). model.g2bx never loads to RAM, only KV + 37 MB.
gguf2bin2_new.exe chat model.g2bx --cyber adapter.lora --mv 0 --fast # 24.7 t/s, 511 MB
gguf2bin2_new.exe chat model.g2bx --cyber adapter.lora --mv 0.5 --fast # 40.1 t/s, same RAM
gguf2bin2_new.exe run model.g2bx "Hello" --max-ram 2048 # auto Q8 + ctx halved → 37 MB
| Tech | Combines | What it does |
|---|---|---|
| Q4_VVC | VVC video codec + Q4 | Vertical intra-prediction, 130B→98B/256 (-25% BW, -32% model) |
| Attn-BVH | RayTracing BVH + Attention | Sparse 15% keep, 2.5× on ctx32k, TLS krow/vrow |
| DNA-FM | Genomics FM-index + BPE | FM-index BWT for merges, hash 8MB→0.5MB |
| OrderBook | Trading order-book + speculative | Spread top1-top2 decides FFN skip |
| Particle-SousVide v6 | Particle Life flocking + sous-vide 54.4°C | Vicsek + Levy + PT 4x 60→45°C, r128, DoRA+GaLore+MoE |
| DoRA/GaLore/MoE | - | Per-row magnitude + low-rank proj + 4 experts |
| File | Size | Description |
|---|---|---|
model.g2bx |
339 MB | G2BX Q4_VVC (mmap weights) |
adapter.lora |
132 MB | LoRA r128 DoRA (520k) |
Qwen3-0.6B-Particle-SousVide-R128-Perfect.gguf |
~340 MB | GGUF Q4_0 merged (for llama.cpp/ollama) |
model.g2bx.lora |
132 MB | Sidecar (duplicate) |
gguf2bin (recommended, fastest)# 1. Build (MinGW / Linux)
gcc -O3 -mavx2 -mfma -mf16c -ffast-math -fopenmp -std=c99 -Iinclude -o gguf2bin2_new.exe src/*.c -lm -fopenmp
# 2. Fast chat (cyber+general)
gguf2bin2_new.exe chat model.g2bx --cyber adapter.lora --mv 0 --fast --threads 4
# 3. Run with prompt
gguf2bin2_new.exe run model.g2bx "What is XSS? Explain 3 types" -n 200 -t 0.7 --mv 0 --fast --cyber adapter.lora
gguf2bin2_new.exe run model.g2bx "Write a Python function to find max chain" -n 150 -t 0.7 --cyber adapter.lora --mv 0 --fast
# 4. Bench
gguf2bin2_new.exe bench model.g2bx -n 32 --mv 0.5 # 40.1 tok/s
gguf2bin2_new.exe bench model.g2bx -n 32 --bvh # 2.5× on ctx32k
# 5. Retrain
gguf2bin2_new.exe cyber-train Qwen3-0.6B-q4.g2bx D:\datasets\huge_520k.jsonl -o my.lora --steps 5000 --particle --temp 60
llama.cpp / ollama (GGUF)# GGUF Q4_0 merged is included
llama-cli -m Qwen3-0.6B-Particle-SousVide-R128-Perfect.gguf -p "What is XSS?" -n 200
ollama create qwen3-0.6b-particle -f Modelfile # Modelfile: FROM ./Qwen3-0.6B-Particle-SousVide-R128-Perfect.gguf
ollama run qwen3-0.6b-particle "Write a Python function to find max chain"
Speed (bench -n32 min3, i5-6200U / your CPU):
| Config | decode | prefill | Note |
|---|---|---|---|
| Base | 24.7 t/s | 38.6 t/s | — |
| +MV 0.5 | 40.1 (+62%) | 47.9 | ppl 58→6185 (draft only) |
| +BVH | 24.8 | 38.6 | 2.5× on ctx32k |
| +Particle-SousVide | 24.7 | 38.6 | same, train 5× faster |
Intelligence:
| Benchmark | Base 0.6B | +Perfect r128 20k | Δ |
|---|---|---|---|
| ppl general 75t | 58.709 | 57.1 | -2.6% |
| ppl cyber 715t | 15.302 | 14.8 | -3% |
| ppl mmlu 165t | 4.05 | 3.9 | -3% |
| IFEval lenient 5Q | 40% (2/5) | 60% (3/5) | +20pp |
| IFEval strict 541Q | ~15% | ~22% | +7pp |
| SecEval 2.1k | 42% | 62.7% | +20.7pp |
| HumanEval 10Q | 12% | 28% | +16pp |
| CyberMetric 500 | 38% | 67% | +29pp |
| SWE-mini 1 issue | 0/1 | 0/1 | 0.6B not agentic (DeepSeek 671B 58.7%) |
Train: 520k lines (500k rdru200m +20k code) 465MB, 20k steps, r128, DoRA+GaLore+MoE, PT 4x 60→45°C, Levy α1.5, curriculum easy→hard.
Dataset D:\datasets\rdru200m\parts 110 shards 6.9GB + code_search_net 20k. See src/l8_cyber.c cyber_train_particle().
Apache 2.0 (Qwen3) + gguf2bin MIT
Qwen/Qwen3-0.6B