VibeThinker-3B-GGUF

Direct GGUF Quantizations of VibeThinker-3B

This repository provides GGUF quantized models for WeiboAI/VibeThinker-3B.

VibeThinker-3B is a powerful 3 billion parameter Small Language Model developed by WeiboAI. Built on the Qwen2.5 architecture, it is fine-tuned for challenging reasoning tasks with clear verification signals, excelling in mathematics, coding, and STEM reasoning. It achieves frontier-level performance on benchmarks like AIME, HMMT, IMO-AnswerBench, and LiveCodeBench, reaching a 96.1% acceptance rate on recent LeetCode weekly/biweekly contests. These GGUF versions are optimized for efficient CPU and GPU inference using llama.cpp and compatible tools.

This release includes various quantization levels (e.g., Q2_K, Q3_K_M, Q4_K_M, Q5_K_M, Q6_K, Q8_0) to suit different hardware capabilities and performance requirements.

Table of Contents 📝

1. ▶ Usage
2. 📃 License
3. 🙏 Acknowledgements

pip install "huggingface_hub[cli]"
huggingface-cli download samuelchristlie/VibeThinker-3B-gguf --local-dir ./VibeThinker-3B-gguf

You can also download directly from this page

2. Inference

To use these GGUF files, you'll need a compatible inference engine like llama.cpp or clients built on top of it (e.g., Ollama, LM Studio, KoboldCpp, text-generation-webui with llama.cpp backend).

Note: VibeThinker-3B was not trained on tool-calling or agent-based programming data. It is best suited for competitive-style math, coding (e.g., LeetCode-style problems), and STEM reasoning tasks. For harder math reasoning, try evaluating against AMOBench with max_tokens set to 60K–100K.

• WeiboAI for developing and open-sourcing the powerful VibeThinker-3B model:
  • VibeThinker-3B on Hugging Face
  • Technical Report (arXiv:2606.16140)
  • GitHub Repository
• The llama.cpp project and its contributors for the GGUF format and the incredible tooling that makes local LLM inference accessible.
  • llama.cpp GitHub Repository