Sign up

khazarai/Qwen3-4B-Gemini-3.1-Pro-Reasoning-Distilled

🤗 Hugging Face sourcetext-generationapache-2.015 GBGGUF✓ HF checksums availableupdated today
No torrent yet →

Qwen3-4B-Gemini-3.1-Pro-Reasoning-Distilled : GGUF

Qwen3-4B-Gemini-3.1-Pro-Reasoning-Distilled is a reasoning-enhanced language model fine-tuned from the Qwen3-4B-Thinking-2507 base model using a high-density synthetic reasoning dataset distilled from Gemini 3.1 Pro.

The goal of this fine-tuning is to improve multi-step reasoning, structured problem decomposition, and analytical thinking while maintaining the strong language capabilities of the base Qwen model.

This model focuses on System-2 style reasoning, emphasizing logical derivations, verification steps, and structured explanations.

  • Model name: khazarai/Qwen3-4B-Gemini-3.1-Pro-Reasoning-Distilled
  • Base model: Qwen3-4B-Thinking-2507

Available Model files:

  • qwen3-4b-thinking-2507.BF16.gguf
  • qwen3-4b-thinking-2507.Q8_0.gguf
  • qwen3-4b-thinking-2507.Q4_K_M.gguf

These files enable deployment in local inference frameworks such as llama.cpp or Ollama.

Training Method

Method:

  • QLoRA (4-bit quantization)
  • LoRA adapters
  • Training framework: Unsloth

Dataset

The model was trained on a synthetic reasoning dataset distilled from Gemini 3.1 Pro.

Dataset Overview

Total Token Volume: 5.6 million tokens The dataset prioritizes logical density and deep reasoning traces, rather than large token counts. The dataset represents a frontier approach in reasoning distillation, where a stronger reasoning model generates structured problem-solving examples that are then used to train a smaller model.

Dataset generation pipeline:

  • Question generation
  • Multi-step reasoning derivation
  • Answer verification

Curated using an agentic workflow involving:

  • Gemini 3 Flash
  • Gemini 3.1 Pro

Domain Coverage

The dataset spans 60+ expert-level domains, focusing on complex reasoning tasks.

  • Physics
  • Mathematics
  • Computer Science
  • Biology & Medicine
  • Strategic Reasoning
  • Advanced Benchmarks

The dataset includes problems inspired by challenging benchmarks such as:

  • ARC-AGI
  • GPQA Diamond
  • MMLU-Pro
  • HLE Benchmark

Intended Use

This model is designed for:

  • Complex reasoning tasks
  • Scientific explanations
  • Advanced mathematics and logic
  • Programming and computer science questions
  • Analytical problem solving

Typical use cases include:

  • research assistance
  • algorithm reasoning
  • technical education
  • complex question answering

Limitations

  • Despite improved reasoning ability, the model still inherits limitations from its base architecture:
  • Hallucinations may still occur in specialized domains.
  • Synthetic reasoning datasets may introduce stylistic biases.
  • Performance may degrade on tasks requiring extremely long context or domain-specific expertise.

Users should verify outputs when used in critical scientific or technical environments.