[!TIP] 🚀 KorshAI-4B-v5 is now available! We strongly recommend the new champion release: KorshLAB/KorshAI-4B-v5 with improved polyglot reasoning and higher benchmark accuracy (Score: 55.85).
KorshAI-4B-v2 is a proprietary, high-performance 4-billion parameter foundation coding model architected and trained by KorshLab and Korsh.
Engineered specifically for polyglot systems programming, modern web architectures, and rigorous formal code verification, KorshAI-4B-v2 delivers enterprise-grade speed and precision on consumer GPUs, workstations, and edge servers.
-Wall -Wextra -Werror -pedantic).llama.cpp CPU/GPU acceleration, delivering upwards of 14-85 tokens/s depending on acceleration tier.| Tier / Platform | Target Setup | Minimum Specs | Memory (RAM / VRAM) | Expected Inference Speed |
|---|---|---|---|---|
| CPU Only (Edge / PC) | Workstations, Laptops, Mini-PC | 4-Core x86_64 CPU (AVX2 support) or AMD Ryzen | 6 GB RAM (active footprint ~3.4 GB) | 5.0 – 15.0 tokens/s (TG) ~45 tok/s (PP) |
| GPU Accelerated | Consumer / Gaming Desktop | NVIDIA GPU (RTX 3060, 3070, 4060, 4070 or better) | 6 GB – 8 GB VRAM (active footprint ~3.6 GB) | 60.0 – 85.0 tokens/s (TG) ~950 tok/s (PP) |
| Apple Silicon | MacBooks, Mac Mini, Mac Studio | Apple M1 / M2 / M3 / M4 (Metal) | 8 GB Unified Memory | 40.0 – 55.0 tokens/s (TG) ~480 tok/s (PP) |
| Low-Power Edge | Single-Board Computers (SBC) | Raspberry Pi 5 / Rockchip RK3588 (ARM64) | 8 GB LPDDR4X | 6.0 – 8.5 tokens/s (TG) |
Memory usage scales predictably with the KV cache allocation according to the target context length:
| Context Window (Tokens) | RAM / VRAM Footprint (Q4_K_M) | Recommended System Memory | Recommended GPU |
|---|---|---|---|
| 2,048 tokens | ~3.1 GB | 6 GB RAM | 4 GB VRAM GPU |
| 4,096 tokens (Standard) | ~3.4 GB | 8 GB RAM | 6 GB VRAM GPU (e.g. RTX 2060, 3060) |
| 8,192 tokens | ~4.1 GB | 8 GB – 12 GB RAM | 6 GB – 8 GB VRAM GPU |
| 16,384 tokens | ~5.5 GB | 12 GB – 16 GB RAM | 8 GB VRAM GPU (e.g. RTX 3070, 4060 Ti) |
| 32,768 tokens | ~8.3 GB | 16 GB+ RAM | 12 GB+ VRAM GPU (e.g. RTX 3060 12GB, 4070) |
llama.cpp (b3000+)ollama (v0.1.30+)LM Studio / Jan.ai / LocalAI / Text Generation WebUIllama-cpp-pythonAll outputs from KorshAI-4B-v2 have undergone automated compilation, execution, and validation against the KorshLab Evaluation Suite:
| Domain / Stack | Test Scope | Verification Standard | Pass Rate |
|---|---|---|---|
| C11 Systems | FP32 / SIMD Dot Product, Bit Manipulation | GCC 12+ (-Wall -Wextra -Werror -pedantic -std=c11) |
100% (PASSED) |
| C++20 Modern | Custom Arena Allocator, RAII Safe Handles | G++ 12+ (-Wall -Wextra -Werror -std=c++20) |
100% (PASSED) |
| PHP 8.2+ Backend | Strict Types (declare(strict_types=1)), OOP, Assertions |
PHP 8.2 CLI Engine | 100% (PASSED) |
| HTML5 & Web | Semantic UI, Accessible Modals, Bootstrap 5 | W3C / WCAG 2.1 AA Compliance Check | 100% (PASSED) |
| Structured Output | Strict JSON Schema Compliance | Python jsonschema RFC 7159 |
100% (PASSED) |
| File | Format | Quantization | Size | Description |
|---|---|---|---|---|
KorshAI-4B-v2.gguf |
GGUF | Q4_K_M (imatrix) | 2.74 GB | Recommended general-purpose quantization with high fidelity and low VRAM footprint (~3.5 GB VRAM recommended). |
llama.cppYou can run KorshAI-4B-v2 directly using llama-cli:
llama-cli -m KorshAI-4B-v2.gguf \
-p "<|im_start|>system\nYou are KorshAI, an expert systems and software engineer.<|im_end|>\n<|im_start|>user\nWrite a thread-safe memory pool in C11.<|im_end|>\n<|im_start|>assistant\n" \
-c 4096 \
-ngl 99 \
--temp 0.2
llama-server)Start an OpenAI-compatible endpoint:
llama-server -m KorshAI-4B-v2.gguf \
--host 0.0.0.0 \
--port 8080 \
-c 8192 \
-ngl 99
Query the model via curl:
curl http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "KorshAI-4B-v2",
"messages": [
{"role": "system", "content": "You are KorshAI, an expert software engineer."},
{"role": "user", "content": "Explain RAII in C++20 with a practical code example."}
],
"temperature": 0.2
}'
Create a file named Modelfile:
FROM ./KorshAI-4B-v2.gguf
TEMPLATE """<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""
SYSTEM """You are KorshAI, a world-class autonomous software engineering model created by KorshLab and korsh.org."""
PARAMETER stop "<|im_end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.2
Build and run with Ollama:
ollama create korshai:4b-v2 -f Modelfile
ollama run korshai:4b-v2
llama-cpp-python)from llama_cpp import Llama
llm = Llama(
model_path="KorshAI-4B-v2.gguf",
n_ctx=4096,
n_gpu_layers=-1
)
response = llm.create_chat_completion(
messages=[
{"role": "system", "content": "You are KorshAI, an expert polyglot programmer."},
{"role": "user", "content": "Implement a binary search tree in C11."}
],
temperature=0.2
)
print(response["choices"][0]["message"]["content"])
KorshAI-4B-v2 is developed and maintained by KorshLab in partnership with the Korsh Foundation.
This model is licensed under the Apache License, Version 2.0. Copyright © 2026 KorshLab & Korsh Foundation. All rights reserved.