OpenVINO IR conversion of deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B with weight-only INT8 quantization, produced on CPU-only Kaggle with optimum-cli (fully offline, token-free).
deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B (Qwen2ForCausalLM, 1.5B params, MIT)text-generation (R1-style reasoning distill on Qwen2.5-1.5B base)optimum-intel 2.2.0, openvino 2026.4.1, transformers 5.16.1<|think|> reasoning format for best resultsfrom optimum.intel import OVModelForCausalLM
from transformers import AutoTokenizer
model = OVModelForCausalLM.from_pretrained("Nekodeus/deepseek-r1-distill-qwen-1-5b-openvino")
tokenizer = AutoTokenizer.from_pretrained("Nekodeus/deepseek-r1-distill-qwen-1-5b-openvino")
inputs = tokenizer("How many primes are there below 100? Let's think step by step.",
return_tensors="pt")
print(tokenizer.decode(model.generate(**inputs, max_new_tokens=128)[0]))
ov::genai::LLMPipeline pipe("path/to/deepseek-r1-distill-qwen-1-5b-openvino", "CPU");
optimum-cli export openvino \
--model deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B \
--task text-generation \
--weight-format int8 \
deepseek-r1-distill-qwen-1.5b-openvino
Note: built with
transformers==5.16.1pinned (Kaggle image ships 5.20.2, which removedVisionRotaryEmbeddingthat optimum-intel 2.2.0 imports).
openvino_model.bin 1.8 GBopenvino_tokenizer.bin 6 MBopenvino_detokenizer.bin 2 MB