[!TIP] 🚀 KorshAI-4B-v5 is now available! We strongly recommend the new champion release: KorshLAB/KorshAI-4B-v5 with improved polyglot reasoning and higher benchmark accuracy (Score: 55.85).
KorshAI-4B-v2 (GGUF)
KorshAI-4B-v2 is a proprietary, high-performance 4-billion parameter foundation coding model architected and trained by KorshLab and Korsh.
Engineered specifically for polyglot systems programming, modern web architectures, and rigorous formal code verification, KorshAI-4B-v2 delivers enterprise-grade speed and precision on consumer GPUs, workstations, and edge servers.
🌟 Key Highlights
- Proprietary Architecture: Native 4B parameter dense architecture tailored for deep polyglot comprehension and algorithmic reasoning.
- Polyglot Mastery: Specialized in low-level systems programming (C11, C++20), modern web stacks (PHP 8.2+, HTML5, Bootstrap 5), Python 3, and structured JSON Schema generation.
- Strict Verification Protocol: Trained and verified using the KorshLab multi-tier sandbox suite with zero-tolerance compiler flags (
-Wall -Wextra -Werror -pedantic). - Optimal Quantization: Quantized with custom importance matrices (imatrix) computed against multi-language source corpora, preserving fp16 perplexity with minimal memory overhead.
- High-Throughput Inference: Optimized for
llama.cppCPU/GPU acceleration, delivering upwards of 14-85 tokens/s depending on acceleration tier.
💻 Hardware Compatibility
1. Requirements Matrix
| Tier / Platform | Target Setup | Minimum Specs | Memory (RAM / VRAM) | Expected Inference Speed |
|---|---|---|---|---|
| CPU Only (Edge / PC) | Workstations, Laptops, Mini-PC | 4-Core x86_64 CPU (AVX2 support) or AMD Ryzen | 6 GB RAM (active footprint ~3.4 GB) | 5.0 – 15.0 tokens/s (TG) ~45 tok/s (PP) |
| GPU Accelerated | Consumer / Gaming Desktop | NVIDIA GPU (RTX 3060, 3070, 4060, 4070 or better) | 6 GB – 8 GB VRAM (active footprint ~3.6 GB) | 60.0 – 85.0 tokens/s (TG) ~950 tok/s (PP) |
| Apple Silicon | MacBooks, Mac Mini, Mac Studio | Apple M1 / M2 / M3 / M4 (Metal) | 8 GB Unified Memory | 40.0 – 55.0 tokens/s (TG) ~480 tok/s (PP) |
| Low-Power Edge | Single-Board Computers (SBC) | Raspberry Pi 5 / Rockchip RK3588 (ARM64) | 8 GB LPDDR4X | 6.0 – 8.5 tokens/s (TG) |
2. Memory Footprint by Context Window
Memory usage scales predictably with the KV cache allocation according to the target context length:
| Context Window (Tokens) | RAM / VRAM Footprint (Q4_K_M) | Recommended System Memory | Recommended GPU |
|---|---|---|---|
| 2,048 tokens | ~3.1 GB | 6 GB RAM | 4 GB VRAM GPU |
| 4,096 tokens (Standard) | ~3.4 GB | 8 GB RAM | 6 GB VRAM GPU (e.g. RTX 2060, 3060) |
| 8,192 tokens | ~4.1 GB | 8 GB – 12 GB RAM | 6 GB – 8 GB VRAM GPU |
| 16,384 tokens | ~5.5 GB | 12 GB – 16 GB RAM | 8 GB VRAM GPU (e.g. RTX 3070, 4060 Ti) |
| 32,768 tokens | ~8.3 GB | 16 GB+ RAM | 12 GB+ VRAM GPU (e.g. RTX 3060 12GB, 4070) |
3. Supported Compute Backends
- CUDA / cuBLAS: Native acceleration on NVIDIA GPUs (compute capability 6.0+).
- Metal: Native hardware acceleration for Apple Silicon macOS devices.
- ROCm / HIP: Native acceleration for supported AMD Radeon GPUs on Linux.
- Vulkan: Universal GPU acceleration across Intel, AMD, and mobile GPUs.
- AVX2 / AVX-512: CPU SIMD vector extensions for high-speed CPU-only inference.
- ARM NEON: Native acceleration for 64-bit ARM architectures (Apple, Raspberry Pi 5, Rockchip).
4. Supported Operating Systems & Runtimes
- Operating Systems: Linux (Ubuntu, Debian, Fedora, Arch, RHEL), Windows 10/11 (MSVC native & WSL2), macOS 12+ (Apple Silicon & Intel).
- Compatible Runtimes:
llama.cpp(b3000+)ollama(v0.1.30+)LM Studio/Jan.ai/LocalAI/Text Generation WebUI- Python bindings via
llama-cpp-python
📊 Benchmark & Verification Results
All outputs from KorshAI-4B-v2 have undergone automated compilation, execution, and validation against the KorshLab Evaluation Suite:
| Domain / Stack | Test Scope | Verification Standard | Pass Rate |
|---|---|---|---|
| C11 Systems | FP32 / SIMD Dot Product, Bit Manipulation | GCC 12+ (-Wall -Wextra -Werror -pedantic -std=c11) |
100% (PASSED) |
| C++20 Modern | Custom Arena Allocator, RAII Safe Handles | G++ 12+ (-Wall -Wextra -Werror -std=c++20) |
100% (PASSED) |
| PHP 8.2+ Backend | Strict Types (declare(strict_types=1)), OOP, Assertions |
PHP 8.2 CLI Engine | 100% (PASSED) |
| HTML5 & Web | Semantic UI, Accessible Modals, Bootstrap 5 | W3C / WCAG 2.1 AA Compliance Check | 100% (PASSED) |
| Structured Output | Strict JSON Schema Compliance | Python jsonschema RFC 7159 |
100% (PASSED) |
📦 Provided Files
| File | Format | Quantization | Size | Description |
|---|---|---|---|---|
KorshAI-4B-v2.gguf |
GGUF | Q4_K_M (imatrix) | 2.74 GB | Recommended general-purpose quantization with high fidelity and low VRAM footprint (~3.5 GB VRAM recommended). |
🚀 Quickstart & Usage
1. Running with llama.cpp
You can run KorshAI-4B-v2 directly using llama-cli:
llama-cli -m KorshAI-4B-v2.gguf \
-p "<|im_start|>system\nYou are KorshAI, an expert systems and software engineer.<|im_end|>\n<|im_start|>user\nWrite a thread-safe memory pool in C11.<|im_end|>\n<|im_start|>assistant\n" \
-c 4096 \
-ngl 99 \
--temp 0.2
2. OpenAI-Compatible API Server (llama-server)
Start an OpenAI-compatible endpoint:
llama-server -m KorshAI-4B-v2.gguf \
--host 0.0.0.0 \
--port 8080 \
-c 8192 \
-ngl 99
Query the model via curl:
curl http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "KorshAI-4B-v2",
"messages": [
{"role": "system", "content": "You are KorshAI, an expert software engineer."},
{"role": "user", "content": "Explain RAII in C++20 with a practical code example."}
],
"temperature": 0.2
}'
3. Ollama Integration
Create a file named Modelfile:
FROM ./KorshAI-4B-v2.gguf
TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""
SYSTEM """You are KorshAI, a world-class autonomous software engineering model created by KorshLab and korsh.org."""
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.2
Build and run with Ollama:
ollama create korshai:4b-v2 -f Modelfile
ollama run korshai:4b-v2
4. Python (llama-cpp-python)
from llama_cpp import Llama
llm = Llama(
model_path="KorshAI-4B-v2.gguf",
n_ctx=4096,
n_gpu_layers=-1
)
response = llm.create_chat_completion(
messages=[
{"role": "system", "content": "You are KorshAI, an expert polyglot programmer."},
{"role": "user", "content": "Implement a binary search tree in C11."}
],
temperature=0.2
)
print(response["choices"][0]["message"]["content"])
🏛️ About Korsh & KorshLab
KorshAI-4B-v2 is developed and maintained by KorshLab in partnership with the Korsh Foundation.
- 🌐 Official Website: https://korsh.org
- 🔬 Laboratory: https://korshlab.space
- 💬 Discord Community: Join our official Discord
📄 License & Attribution
This model is licensed under the Apache License, Version 2.0. Copyright © 2026 KorshLab & Korsh Foundation. All rights reserved.