JiRack Ultra 14B (CPU)
A fast and efficient 14B model optimized for CPU inference. The model was refactored with BitNet features and an updated tokenizer that includes new Routing, Media, Vision, Sound, Tool call, and Robotics tags. Built on a DeepSeek model redesigned by CMSManhattan with native ternary (BitNet-style) support and ready-to-run GGUF quantizations.
- JiRack is a cloud-ready model that helps save money on cloud infrastructure. It can be used as an expert model in RAG deployments, with the ONNX JiRack Java server as an alternative.
JiRack Ternary Architedure & JiRack Tokenizer
- Benefits high quality CPU inference TQ_2 on Llama.cpp and Ollama via QAT
- Robotcs, Routing, Coding, Multimedia, Advanced tool calling via CMSManhattan/JiRackPrecisionTokenizer
PARTNERSHIP
JiRack sevice options
- Current quantizations were done from the FP16 model, but the model allows for more compression thanks to its ternary architecture.
- If you need to do ternary compression, please write to me and I'll perform QAT from your dataset, tailored specifically to your task.
- Plus double QAT via ONNX QAT.
- Adapt train process to avoid catastrophic forgetting with NDA
- Adapt train process to avoid fast plato in training with NDA
- Convert model to TQ2_0 with support AVX2 and AVX-512 CPU instructions for high performance on CPU
- QAT for TQ_2 Llama.cpp Ternarization docs https://huggingface.co/CMSManhattan/JiRackUltra_14b/blob/main/QAT_to_Llama.cpp_GGUF_TQ2_0_JirackUltra_14b.md
- Adapts to agentic or instruct models for tool calling, using the JiRak tokenizer to enable high-quality tool calling on small models — built as a domain-specific tool expert.
- Deployment and scale
- JiRack Ternary production demo: https://huggingface.co/CMSManhattan/JiRackTernaryPrecision-16B-chat
QAT
- QAT approach 1 script train_qat_ultra_14b_bf16_fulltext_ada_warmup.py
- QAT approach 2 script train_qat_ultra_14b_bf16_with_mask_ada_warmup.py
- The folder for QAT datasets from NVIDIA Nemotron with Open Assistant tokenizer, CMSManhattan/JiRackUltra_14b/prepared_sft_data_ultra14b/
Live Chat
https://huggingface.co/spaces/CMSManhattan/JiRackUltra_14b
Live chat TQ2_O quants
JiRack Codding Agent IDE
JiRack Codding Agent IDE files
Ollama production support
- We are working to support JiRack on Ollama for production systems also
- added Jirack chat without reasoning feature https://ollama.com/cmsmanhattan
- Follow fresh Ollama platform updates
Spring Boot AI tool calls examples for JiRack Ultra series
GoEx AI tool calls examples for JiRack Ultra series
JiRack Ultra 1 tool calls to boost tool call quality
JiRack RoboTech
Export to ONNX
Available Variants
Quick Start
Run with Docker
- 14 B docker can be provided by request .
- Build docker on local from source or request fro me
Default CPU (Q4 recommended)
docker run -d \
--name jirack_ultra_14b \
-p 7869:7869 \
--restart unless-stopped \
cmsmanhattan/jirack-ultra-14b-cpu-q4:latest
Q3
docker run -d \
--name jirack_ultra_14b \
-p 7869:7869 \
--cpus=16 \
-e THREADS=16 \
-e THREADS_BATCH=16 \
--restart unless-stopped \
cmsmanhattan/jirack-ultra-14b-cpu-q3:latest
Q2 (lowest memory)
docker run -d \
--name jirack_ultra_14b \
-p 7869:7869 \
--cpus=16 \
-e THREADS=16 \
-e THREADS_BATCH=16 \
--restart unless-stopped \
cmsmanhattan/jirack-ultra-14b-cpu-q2:latest
Full precision
docker run -d \
--name jirack_ultra_14b \
-p 7869:7869 \
--cpus=16 \
-e THREADS=16 \
-e THREADS_BATCH=16 \
--restart unless-stopped \
cmsmanhattan/jirack-ultra-14b-cpu:latest
Multi CPU
docker run -d \
--name jirack_ultra_14b \
-p 7869:7869 \
--cpus=16 \
-e THREADS=16 \
-e THREADS_BATCH=16 \
--restart unless-stopped \
--memory=16g \
--cpus=8 \
cmsmanhattan/jirack-ultra-14b-cpu-q4:latest
Docker Compose Example
services:
jirack:
image: cmsmanhattan/jirack-ultra-14b-cpu-q4:latest
container_name: jirack_ultra_14b
ports:
- "7869:7869"
volumes:
- .:/app
- ./web:/app/web
environment:
- MAX_TOKENS=2048
- TEMPERATURE=0.7
- TOP_P=0.9
- DEFAULT_STREAM=False
- INTRA_THREADS=4
- USE_ENV_ALLOCATOR=1
- THREADS=16
- THREADS_BATCH=16
deploy:
resources:
limits:
memory: 16g
Access the UI
Once the container is running, open your browser and navigate to:
http://localhost:7869
This opens the JiRack UI — a clean web interface.
Changing the Port
The listening port can be easily modified directly from the Settings panel within the JiRack UI.
Licensing
- Model weights are released under the MIT License — free to use, modify, and distribute for any purpose, including commercial. No royalties, no per-user fees, no subscription.
- The Docker image with UI and the pre-built Ollama quantizations are separate paid products. If you prefer to build your own secure deployment — take the weights, assemble your own stack, and you're done.
- The JiRack Ultra 14B model for Docker and Ollama is provided under a commercial license ($12 per user per year).
- All JiRack UI clients are provided under a commercial license.
- However, the UI clients can be used for free when running together with the official JiRack Docker containers, as long as they are not redistributed separately.
For commercial licensing, cluster deployment, or enterprise use of JiRack models, please contact us.
- JiRack MS Windows 11 Desktop Client (with Ollama API):
https://huggingface.co/kgrabko/JiRackTernary_1b/resolve/main/jirack-chat.zip
- Live email chat with the model: support@cmsmanhattan.com
Hardware Recommendations
Recommended Hardware for JiRack Ultra 14B (single Docker container)
Important Memory Notes
Even though the quantized 14B models are relatively compact, we recommend the following for best experience:
- Q4_K_M: 16–24 GB system RAM minimum
- Q3_K_M / Q2_K: 12–20 GB system RAM
- Full precision: 32 GB+ system RAM recommended
Reasons for extra headroom:
- KV-cache consumption during generation
- Runtime overhead and temporary buffers
- System stability and avoiding out-of-memory errors
- Room for larger context windows
Minimum recommended (Q4): 16 GB system RAM
Ideal: 24–32 GB system RAM
I added the default model in full precision. This serves as the base for quantization, allowing us to find the optimal balance between model size and performance.
Architecture Notes
- Refactored with BitNet features: Native BitLinear ternary path (b1.58-style) with λ-warmup STE
- Updated tokenizer: Extended with new special tags for Routing, Tool call, and Robotics
- Architecture: Qwen2 layout (hidden 5,120, intermediate 13,824, 48 layers, GQA 40/8, vocabulary 152,064, untied embeddings, RoPE theta 1,000,000)
- RoPE θ = 1 000 000, RMSNorm ε = 1e-6
- Intermediate size: 13824
- Ready-to-run GGUF quantizations (Q2_K, Q3_K_M, Q4_K_M)
ROBOS HUMANOID Project
📧 Contact & Licensing
For joint venture opportunities, hardware integration, or licensing inquiries:
License
MIT License
Base models: a DeepSeek model (MIT License, LICENSE-DEEPSEEK), itself derived from a Qwen model (Apache License 2.0, LICENSE-APACHE); see NOTICE.md.
