Granite-3.2-2B-ArkCompact-1.58bit

Sovereign 1.58-Bit Base-3 Ternary Model (2.53B) — Powered by ArkheionNet & ArkCompact

IBM Granite 3.2 2B quantized to sovereign 1.58-bit Base-3 ternary format with ArkCompact. Holds the world-record inference throughput of 98,239 tokens/sec on consumer GPU hardware.

---

⚡ Verified Hardware Performance (AMD Radeon RX 6600M & RDNA2)

---

🏛️ Mathematical Quantization Architecture

ArkCompact quantizes weights into ternary states using Base-3 Packing (5 trits per byte): $$w_{i,j} \in {-\alpha, 0, +\alpha}, \quad 3^5 = 243 \le 256$$

This eliminates 16-bit floating-point multiplications, replacing them with integer accumulations and fused Wave32 bitwise masks:

• Zero-Copy Memory-Mapped Loading (mmap): Model initializes in $< 450\text{ ms}$.
• Multi-Head Latent Attention (MLA): KV-Cache footprint reduced by $-85.9%$.
• Chunked Prefill: Eliminates Head-of-Line blocking in continuous batching.

---

🚀 Quickstart & Inference

1. Run natively with ark-engine (Rust Server)

# Clone and build ArkheionNet
git clone https://github.com/Arkheion/ArkheionNet.git
cd ArkheionNet
cargo build --release -p ark-engine

# Start sovereign server on port 11500
./target/release/ark-engine server --port 11500 --model output/granite-3.2-2b.ark

2. Query via OpenAI-Compatible REST API

curl http://localhost:11500/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Granite-3.2-2B-ArkCompact-1.58bit",
    "messages": [
      {"role": "user", "content": "Explain quantum decoherence in simple terms."}
    ],
    "temperature": 0.2,
    "max_tokens": 512
  }'

3. Use via ark-sdk (Rust)

use ark_sdk::ArkClient;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let client = ArkClient::new("http://localhost:11500");
    let mut stream = client.generate_stream("Granite-3.2-2B-ArkCompact-1.58bit", "Hello Arkheion!").await?;
    
    while let Some(chunk) = stream.next().await {
        print!("{}", chunk?.response);
    }
    Ok(())
}

---

📄 License & Attribution

• Base Model: ibm-granite/granite-3.2-2b-instruct
• Quantization & Runtime: Arkheion Sovereign AI Infrastructure (Apache-2.0)