🗿⚛️ Deep Thought 4B - Science, Mathematics, Drug Discovery & Advanced Reasoning

DON'T PANIC. Welcome to Deep Thought, ostensibly the second greatest computational matrix in the universe. While its predecessor spent 7.5 million years just to output "42," this modern iteration is heavily fine-tuned on the galaxy’s most rigorous STEM and hard science datasets, powered by a vastly upgraded reasoning matrix. It can unravel the laws of the cosmos in seconds-though it will still profoundly judge you for asking it to code a basic HTML to-do list. Grab your towel and behold an AI that might actually figure out the Ultimate Question.

🚀 About

Deep Thought 4B is a 16-bit LoRA fine-tune of Qwen3.5-4B, specialized in advanced reasoning, mathematics, scientific problem-solving, and technical knowledge.

It builds on the excellent Jackrong/DeepSeek-V4-Pro-Qwen3.5-4B, using its uncensored counterpart, xtools-at/DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic, as the starting checkpoint.

The goal is straightforward: take an already capable reasoning model and push it further in the areas where rigorous thinking and specialized knowledge matter most.

Its primary areas of focus are:

• Mathematics: advanced problem-solving, proofs, algebra, calculus, and quantitative reasoning.
• Science: physics, chemistry, biology, and PhD-level scientific questions.
• STEM: engineering, technical analysis, and multilingual scientific knowledge.
• Drug discovery: pharmaceutical research, molecular reasoning, and formulation development.
• Advanced reasoning: structured problem decomposition, multi-step analysis, and answer construction.

Looking for a smarter version? Check out xtools-at/Qwen3.5-9B-DeepThought-STEM

Looking for quantizations/GGUFs? Click here

🧬 Model lineage

Deep Thought inherits its architecture from Qwen3.5-4B, a 4-billion-parameter multimodal foundation model from the Qwen team.

Its reasoning lineage comes from Jackrong's DeepSeek-V4-Pro-Qwen3.5-4B, a Qwen3.5-9B fine-tune distilled from responses generated by DeepSeek-V4-Pro in Max Effect mode.

The Jackrong model was trained on approximately 250,000 mathematics and STEM samples using supervised fine-tuning, followed by a light reinforcement-learning refinement stage using GSPO. This is the upstream model's training history, not Deep Thought's own training run.

Deep Thought builds on this foundation with additional scientific domain knowledge, pharmaceutical research data, synthetic reasoning traces, and a small calibration mixture.

The additional fine-tuning data described below is text-based; it does not represent a dedicated vision-training stage.

📚 Training data

The training mixture contains approximately 11,000+ examples, organized into three complementary groups. An additional 5% evaluation set has been sampled from each group.

1. Reasoning - 6,000+ examples

A combination of frontier-model reasoning distillations and mathematics-focused training data.

The four frontier-model datasets collectively contribute ~4,800 examples.

The Luna source contains an author-reported 100k+ conversations. Deep Thought uses a sample from selected categories rather than applying the entire dataset.

The mixture is intended to expose the model to different reasoning styles and approaches instead of relying on a single teacher model.

2. Domain knowledge - 4,000 examples

Additional training focused on scientific depth and specialized technical knowledge.

The GLM-derived science and STEM subsets originate from reasoning traces distilled from GLM-5.1, originally released in the Kassadin88/GLM-5.1-1000000x dataset and subsequently cleaned and reformatted by Jackrong.

The pharmaceutical component is derived from RED-PILL: Research Enhanced Dataset for Pharmaceutical Innovation in Learning & Language.

Its underlying sources include ChEMBL 35, PubMed, arXiv, ClinicalTrials.gov, the FDA Orange Book, and expert-generated formulation and drug-discovery knowledge bases.

3. Calibration - 1,000 examples

A smaller calibration stage balances explicit reasoning with direct answers and structured instruction following.

These subsets come from Hugging Face's SmolTalk2 post-training mixture, developed for SmolLM3-3B.

The science subset is derived from Open R1's Mixture-of-Thoughts, with reasoning traces removed. The table subset originates from Table-GPT. The multi-turn reasoning component provides additional conversational supervision with explicit reasoning traces.

⚙️ Training configuration

Deep Thought uses LoRA fine-tuning with 16-bit adapter weights. Details:

Training results

Training and evaluation losses of the training run:

📊 Preliminary evaluation

Due to resource constraints, this initial evaluation was limited to the 4B variant, comparing it to the upstream parent model, using a small set of benchmarks. The results suggest that fine-tuning has largely preserved general benchmark performance, with no clear indication of overall degradation. Scores shifted slightly across tasks, with Deep Thought performing slightly better on both ARC benchmarks and lower on HellaSwag and GSM8K.

This is a preliminary assessment rather than a comprehensive evaluation. Testing may be expanded in the future, including runs on the 9B model and additional specialized STEM benchmarks.

Results

Best score per task is in bold. Both models loaded with BNB 4-bit quantization. Full test splits were used for HellaSwag, ARC-Easy, ARC-Challenge and GSM8K.

⚠️ Limitations and intended use

Deep Thought is intended for research, education, scientific exploration, and technical problem-solving.

Despite its specialized training, it can still produce incorrect calculations, misleading explanations, fabricated references, or scientifically plausible but invalid conclusions. Its knowledge may also be incomplete or outdated.

Drug discovery and pharmaceutical formulation are particularly demanding areas. Model outputs should be treated as research assistance, not validated experimental protocols, clinical recommendations, or substitutes for qualified scientific and medical expertise. Important claims should be independently checked against primary literature, authoritative databases, and experimental evidence.

📜 Licenses and attribution

Deep Thought has multiple upstream dependencies, so its licensing should not be assumed to be identical to that of its original base model.

• Qwen3.5-4B: released under Apache License 2.0.
• Jackrong/DeepSeek-V4-Pro-Qwen3.5-4B: declares Apache License 2.0.
• xtools-at/DeepSeek-V4-Pro-Qwen3.5-4B-UNCENSORED-heretic: declares Apache License 2.0.
• GLM-5.1-Reasoning-1M-Cleaned: declares Apache License 2.0 and is derived from Kassadin88/GLM-5.1-1000000x. Its GLM-5.1 teacher model declares MIT. Original dataset terms should also be reviewed.
• RED-PILL: declares Apache License 2.0.
• SmolTalk2: the newly created multi-turn-reasoning-if dataset is listed under Apache 2.0. Existing public components retain their original licenses. In particular, Table-GPT declares MIT, while the Mixture-of-Thoughts repository should be checked for its applicable distribution terms.
• GPT-5.6 Luna reasoning: Hugging Face metadata declares MIT. The author describes the corpus as independently generated using paid OpenAI API inference. The repository's MIT declaration does not, by itself, establish that all applicable provider terms and downstream model-training permissions have been satisfied.
• Claude Haiku 5.5 reasoning, Xiaomi MiMo 2.6 Pro reasoning, and ChatGPT Terra 6.1 reasoning: check each repository's license declaration and the relevant teacher-provider terms before redistribution.

---

Thanks to the Qwen team, Jackrong, Hugging Face, the original dataset authors, and the researchers and model developers whose work makes projects like Deep Thought possible.

🧣 Remember to bring your towel!