DeepSeek-R1-0528-Qwen3-8B-GGUF

Direct GGUF Quantizations of DeepSeek-R1-0528-Qwen3-8B

This repository provides GGUF quantized models for deepseek-ai/DeepSeek-R1-0528-Qwen3-8B.

DeepSeek-R1-0528-Qwen3-8B is a powerful 8 billion parameter Large Language Model developed by DeepSeek AI. It is an Instruct model based on the Qwen3 architecture, excelling in a wide range of text generation tasks including chat, coding, and reasoning. These GGUF versions are optimized for efficient CPU and GPU inference using llama.cpp and compatible tools.

This release includes various quantization levels (e.g., Q2_K, Q3_K_M, Q4_K_M, Q5_K_M, Q6_K, Q8_0) to suit different hardware capabilities and performance requirements.

Table of Contents 📝

1. ▶ Usage
2. 📃 License
3. 🙏 Acknowledgements

pip install "huggingface_hub[cli]"
huggingface-cli download samuelchristlie/DeepSeek-R1-0528-Qwen3-8B-GGUF --local-dir ./DeepSeek-R1-0528-Qwen3-8B-GGUF

You can also download directly from this page

2. Inference

To use these GGUF files, you'll need a compatible inference engine like llama.cpp or clients built on top of it (e.g., Ollama, LM Studio, KoboldCpp, text-generation-webui with llama.cpp backend).

• DeepSeek AI for developing and open-sourcing the powerful DeepSeek-R1-0528-Qwen3-8B model:
  • DeepSeek-R1-0528-Qwen3-8B on Hugging Face
• The llama.cpp project and its contributors for the GGUF format and the incredible tooling that makes local LLM inference accessible.
  • llama.cpp GitHub Repository
• city96:
  • https://huggingface.co/city96