Qwen3-4B-Gemini-3.1-Pro-Reasoning-Distilled : GGUF
Qwen3-4B-Gemini-3.1-Pro-Reasoning-Distilled is a reasoning-enhanced language model fine-tuned from the Qwen3-4B-Thinking-2507 base model using a high-density synthetic reasoning dataset distilled from Gemini 3.1 Pro.
The goal of this fine-tuning is to improve multi-step reasoning, structured problem decomposition, and analytical thinking while maintaining the strong language capabilities of the base Qwen model.
This model focuses on System-2 style reasoning, emphasizing logical derivations, verification steps, and structured explanations.
- Model name:
khazarai/Qwen3-4B-Gemini-3.1-Pro-Reasoning-Distilled - Base model:
Qwen3-4B-Thinking-2507
Available Model files:
qwen3-4b-thinking-2507.BF16.ggufqwen3-4b-thinking-2507.Q8_0.ggufqwen3-4b-thinking-2507.Q4_K_M.gguf
These files enable deployment in local inference frameworks such as llama.cpp or Ollama.
Training Method
Method:
- QLoRA (4-bit quantization)
- LoRA adapters
- Training framework: Unsloth
Dataset
The model was trained on a synthetic reasoning dataset distilled from Gemini 3.1 Pro.
Dataset Overview
Total Token Volume: 5.6 million tokens The dataset prioritizes logical density and deep reasoning traces, rather than large token counts. The dataset represents a frontier approach in reasoning distillation, where a stronger reasoning model generates structured problem-solving examples that are then used to train a smaller model.
Dataset generation pipeline:
- Question generation
- Multi-step reasoning derivation
- Answer verification
Curated using an agentic workflow involving:
- Gemini 3 Flash
- Gemini 3.1 Pro
Domain Coverage
The dataset spans 60+ expert-level domains, focusing on complex reasoning tasks.
- Physics
- Mathematics
- Computer Science
- Biology & Medicine
- Strategic Reasoning
- Advanced Benchmarks
The dataset includes problems inspired by challenging benchmarks such as:
- ARC-AGI
- GPQA Diamond
- MMLU-Pro
- HLE Benchmark
Intended Use
This model is designed for:
- Complex reasoning tasks
- Scientific explanations
- Advanced mathematics and logic
- Programming and computer science questions
- Analytical problem solving
Typical use cases include:
- research assistance
- algorithm reasoning
- technical education
- complex question answering
Limitations
- Despite improved reasoning ability, the model still inherits limitations from its base architecture:
- Hallucinations may still occur in specialized domains.
- Synthetic reasoning datasets may introduce stylistic biases.
- Performance may degrade on tasks requiring extremely long context or domain-specific expertise.
Users should verify outputs when used in critical scientific or technical environments.