Base model: deepseek-ai/DeepSeek-V3.2-Speciale
Note:
1. Tested on Hopper device, we don't know if
ada / ampere devices could run this repo yet.
2. Waiting for official chat_template.jinja;
The file in this repo is borrowed from v3.1
3. This is a thinking model, make sure to include
extra_body = {"chat_template_kwargs": {"thinking": True}}
in the post requests, in order to have thinking blocks properly parsed.
As of 2025-12-02, make sure your system has cuda12.8 installed.
Then, create a fresh Python environment (e.g. python3.12 venv) and run:
# install vllm
pip install vllm==0.11.2
# install deep_gemm
git clone https://github.com/deepseek-ai/DeepGEMM.git
cd DeepGEMM/third-party
git clone https://github.com/NVIDIA/cutlass.git
git clone https://github.com/fmtlib/fmt.git
cd ../
git checkout v2.1.1.post3
pip install . --no-build-isolation
or
uv pip install vllm --extra-index-url https://wheels.vllm.ai/nightly
uv pip install git+https://github.com/deepseek-ai/DeepGEMM.git@v2.1.1.post3 --no-build-isolation # Other versions may also work. We recommend using the latest released version from https://github.com/deepseek-ai/DeepGEMM/releases
see Official vLLM Deepseek-V3.2 Guide
Note: It could take a little while to load, if --enable-expert-parallel is enabled;
export VLLM_USE_DEEP_GEMM=0 # ATM, this line is a "must" for Hopper devices
export TORCH_ALLOW_TF32_CUBLAS_OVERRIDE=1
export VLLM_USE_FLASHINFER_MOE_FP16=1
export VLLM_USE_FLASHINFER_SAMPLER=0
export OMP_NUM_THREADS=4
CONTEXT_LENGTH=32768
vllm serve \
__YOUR_PATH__/QuantTrio/DeepSeek-V3.2-Speciale-AWQ \
--served-model-name MY_MODEL_NAME \
--enable-auto-tool-choice \
--tool-call-parser deepseek_v31 \
--reasoning-parser deepseek_v3 \
--swap-space 16 \
--max-num-seqs 32 \
--max-model-len $CONTEXT_LENGTH \
--gpu-memory-utilization 0.9 \
--tensor-parallel-size 8 \
--enable-expert-parallel \ # optional
--speculative-config '{"model": "__YOUR_PATH__/QuantTrio/DeepSeek-V3.2-Speciale-AWQ", "num_speculative_tokens": 1}' \ # optional, 50%+- throughput increase is observed
--trust-remote-code \
--host 0.0.0.0 \
--port 8000
2025-12-04
1. Initial commit
| File Size | Last Updated |
|---|---|
338 GiB |
2025-12-04 |
from huggingface_hub import snapshot_download
snapshot_download('QuantTrio/DeepSeek-V3.2-Speciale-AWQ', cache_dir="your_local_path")
We introduce DeepSeek-V3.2, a model that harmonizes high computational efficiency with superior reasoning and agent performance. Our approach is built upon three key technical breakthroughs:
We have also released the final submissions for IOI 2025, ICPC World Finals, IMO 2025 and CMO 2025, which were selected based on our designed pipeline. These materials are provided for the community to conduct secondary verification. The files can be accessed at assets/olympiad_cases.
DeepSeek-V3.2 introduces significant updates to its chat template compared to prior versions. The primary changes involve a revised format for tool calling and the introduction of a "thinking with tools" capability.
To assist the community in understanding and adapting to this new template, we have provided a dedicated encoding folder, which contains Python scripts and test cases demonstrating how to encode messages in OpenAI-compatible format into input strings for the model and how to parse the model's text output.
A brief example is illustrated below:
import transformers
# encoding/encoding_dsv32.py
from encoding_dsv32 import encode_messages, parse_message_from_completion_text
tokenizer = transformers.AutoTokenizer.from_pretrained("deepseek-ai/DeepSeek-V3.2")
messages = [
{"role": "user", "content": "hello"},
{"role": "assistant", "content": "Hello! I am DeepSeek.", "reasoning_content": "thinking..."},
{"role": "user", "content": "1+1=?"}
]
encode_config = dict(thinking_mode="thinking", drop_thinking=True, add_default_bos_token=True)
# messages -> string
prompt = encode_messages(messages, **encode_config)
# Output: "<|begin▁of▁sentence|><|User|>hello<|Assistant|></think>Hello! I am DeepSeek.<|end▁of▁sentence|><|User|>1+1=?<|Assistant|><think>"
# string -> tokens
tokens = tokenizer.encode(prompt)
# Output: [0, 128803, 33310, 128804, 128799, 19923, 3, 342, 1030, 22651, 4374, 1465, 16, 1, 128803, 19, 13, 19, 127252, 128804, 128798]
Important Notes:
developer has been introduced in the chat template. This role is dedicated exclusively to search agent scenarios and is designated for no other tasks. The official API does not accept messages assigned to developer.The model structure of DeepSeek-V3.2 and DeepSeek-V3.2-Speciale are the same as DeepSeek-V3.2-Exp. Please visit DeepSeek-V3.2-Exp repo for more information about running this model locally.
Usage Recommendations:
temperature = 1.0, top_p = 0.95.This repository and the model weights are licensed under the MIT License.
@misc{deepseekai2025deepseekv32,
title={DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models},
author={DeepSeek-AI},
year={2025},
}
If you have any questions, please raise an issue or contact us at service@deepseek.com.
model-00005-of-00121.safetensors 3.0 GBmodel-00098-of-00121.safetensors 3.0 GBmodel-00112-of-00121.safetensors 3.0 GBmodel-00072-of-00121.safetensors 3.0 GBmodel-00088-of-00121.safetensors 3.0 GBmodel-00016-of-00121.safetensors 3.0 GBmodel-00102-of-00121.safetensors 3.0 GBmodel-00116-of-00121.safetensors 3.0 GBmodel-00110-of-00121.safetensors 3.0 GBmodel-00086-of-00121.safetensors 3.0 GBmodel-00096-of-00121.safetensors 3.0 GBmodel-00070-of-00121.safetensors 3.0 GBmodel-00104-of-00121.safetensors 3.0 GBmodel-00080-of-00121.safetensors 3.0 GBmodel-00054-of-00121.safetensors 3.0 GBmodel-00106-of-00121.safetensors 3.0 GBmodel-00064-of-00121.safetensors 3.0 GBmodel-00090-of-00121.safetensors 3.0 GBmodel-00038-of-00121.safetensors 3.0 GBmodel-00094-of-00121.safetensors 3.0 GBmodel-00062-of-00121.safetensors 3.0 GBmodel-00108-of-00121.safetensors 3.0 GBmodel-00030-of-00121.safetensors 3.0 GBmodel-00046-of-00121.safetensors 3.0 GBmodel-00048-of-00121.safetensors 3.0 GBmodel-00078-of-00121.safetensors 3.0 GBmodel-00092-of-00121.safetensors 3.0 GBmodel-00074-of-00121.safetensors 3.0 GBmodel-00100-of-00121.safetensors 3.0 GBmodel-00114-of-00121.safetensors 3.0 GBmodel-00022-of-00121.safetensors 3.0 GBmodel-00032-of-00121.safetensors 3.0 GBmodel-00052-of-00121.safetensors 3.0 GBmodel-00058-of-00121.safetensors 3.0 GBmodel-00068-of-00121.safetensors 3.0 GBmodel-00084-of-00121.safetensors 3.0 GBmodel-00036-of-00121.safetensors 3.0 GBmodel-00020-of-00121.safetensors 3.0 GBmodel-00056-of-00121.safetensors 3.0 GBmodel-00042-of-00121.safetensors 3.0 GBmodel-00066-of-00121.safetensors 3.0 GBmodel-00076-of-00121.safetensors 3.0 GBmodel-00026-of-00121.safetensors 3.0 GBmodel-00040-of-00121.safetensors 3.0 GBmodel-00050-of-00121.safetensors 3.0 GBmodel-00060-of-00121.safetensors 3.0 GBmodel-00044-of-00121.safetensors 3.0 GBmodel-00034-of-00121.safetensors 3.0 GBmodel-00024-of-00121.safetensors 3.0 GBmodel-00028-of-00121.safetensors 3.0 GBmodel-00018-of-00121.safetensors 3.0 GBmodel-00004-of-00121.safetensors 3.0 GBmodel-00082-of-00121.safetensors 3.0 GBmodel-00014-of-00121.safetensors 3.0 GBmodel-00010-of-00121.safetensors 3.0 GBmodel-00008-of-00121.safetensors 3.0 GBmodel-00012-of-00121.safetensors 3.0 GBmodel-00002-of-00121.safetensors 3.0 GBmodel-00081-of-00121.safetensors 3.0 GBmodel-00019-of-00121.safetensors 3.0 GBmodel-00025-of-00121.safetensors 3.0 GBmodel-00029-of-00121.safetensors 3.0 GBmodel-00035-of-00121.safetensors 3.0 GBmodel-00041-of-00121.safetensors 3.0 GBmodel-00045-of-00121.safetensors 3.0 GBmodel-00051-of-00121.safetensors 3.0 GBmodel-00057-of-00121.safetensors 3.0 GBmodel-00061-of-00121.safetensors 3.0 GBmodel-00067-of-00121.safetensors 3.0 GBmodel-00073-of-00121.safetensors 3.0 GBmodel-00027-of-00121.safetensors 3.0 GBmodel-00077-of-00121.safetensors 3.0 GBmodel-00017-of-00121.safetensors 3.0 GBmodel-00083-of-00121.safetensors 3.0 GBmodel-00089-of-00121.safetensors 3.0 GBmodel-00043-of-00121.safetensors 3.0 GBmodel-00021-of-00121.safetensors 3.0 GBmodel-00093-of-00121.safetensors 3.0 GBmodel-00099-of-00121.safetensors 3.0 GBmodel-00113-of-00121.safetensors 3.0 GBmodel-00033-of-00121.safetensors 3.0 GBmodel-00037-of-00121.safetensors 3.0 GBmodel-00053-of-00121.safetensors 3.0 GBmodel-00069-of-00121.safetensors 3.0 GBmodel-00085-of-00121.safetensors 3.0 GBmodel-00101-of-00121.safetensors 3.0 GBmodel-00107-of-00121.safetensors 3.0 GBmodel-00115-of-00121.safetensors 3.0 GBmodel-00059-of-00121.safetensors 3.0 GBmodel-00023-of-00121.safetensors 3.0 GBmodel-00047-of-00121.safetensors 3.0 GBmodel-00079-of-00121.safetensors 3.0 GBmodel-00031-of-00121.safetensors 3.0 GBmodel-00063-of-00121.safetensors 3.0 GBmodel-00095-of-00121.safetensors 3.0 GBmodel-00109-of-00121.safetensors 3.0 GBmodel-00049-of-00121.safetensors 3.0 GBmodel-00075-of-00121.safetensors 3.0 GBmodel-00039-of-00121.safetensors 3.0 GBmodel-00065-of-00121.safetensors 3.0 GBmodel-00091-of-00121.safetensors 3.0 GBmodel-00055-of-00121.safetensors 3.0 GBmodel-00105-of-00121.safetensors 3.0 GBmodel-00103-of-00121.safetensors 3.0 GBmodel-00003-of-00121.safetensors 3.0 GBmodel-00009-of-00121.safetensors 3.0 GBmodel-00013-of-00121.safetensors 3.0 GBmodel-00007-of-00121.safetensors 3.0 GBmodel-00011-of-00121.safetensors 3.0 GBmodel-00015-of-00121.safetensors 3.0 GBmodel-00117-of-00121.safetensors 3.0 GBmodel-00087-of-00121.safetensors 3.0 GBmodel-00120-of-00121.safetensors 3.0 GBmodel-00006-of-00121.safetensors 3.0 GBmodel-00118-of-00121.safetensors 3.0 GBmodel-00001-of-00121.safetensors 3.0 GBmodel-00071-of-00121.safetensors 3.0 GBmodel-00097-of-00121.safetensors 3.0 GBmodel-00111-of-00121.safetensors 2.9 GBmodel-00119-of-00121.safetensors 2.9 GBmodel-00121-of-00121.safetensors 2.3 GBHave the files? Submit a torrent.
Submit a torrentHave the complete files and permission to share them? Replace the example folder path. The script checks the files, makes a torrent and submits its magnet and checksums. Keep your torrent client seeding. Get your community key from your settings, and read the script before running it.
curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo QuantTrio/DeepSeek-V3.2-Speciale-AWQ ./model-folderdeepseek-ai/DeepSeek-V3.2-Speciale