💊 Red Pill V1.1
Pharmaceutical AI That Won't Say No
Fine-tuned for drug discovery & formulation development
What is Red Pill?
Red Pill is a 1B-parameter language model fine-tuned on the RED-PILL dataset — 1,047 multi-turn pharmaceutical instructions covering drug discovery, formulation development, synthesis, regulatory affairs, and scientific literature.
Built on the Heretic-abliterated MiniCPM5-1B base, Red Pill won't refuse legitimate pharmaceutical R&D questions — from molecular design to formulation strategy to regulatory pathways.
"In the pharmaceutical world, the model that says 'I can't help with that' is the model that gets replaced."
🧪 Capabilities
| Domain | What Red Pill Can Do |
|--------|---------------------|
| Drug Discovery | Hit-to-lead optimization, ADMET profiling, virtual screening strategy, SAR analysis |
| Formulation | Solubility enhancement (ASD, nanosizing, cyclodextrin), dosage form design, excipient selection |
| Synthesis | Retrosynthetic analysis, process chemistry, scale-up considerations |
| Regulatory | ANDA vs 505(b)(2) pathways, IND-enabling studies, FDA guidance |
| Decision Making | Trade-off analysis, prioritization frameworks, risk assessment |
| Literature | PubMed paper summarization, method critique, cross-paper synthesis |
📊 Evaluation
Tested on 32 expert pharmaceutical questions (Kaggle, 2× T4 GPU):
Results by Category
| Category | Score | Status |
|----------|-------|--------|
| Decision Making | 41.4% | 🟢 Strong |
| Formulation Design | 24.8% | 🟡 Developing |
| Molecular Analysis | 22.6% | 🟡 Developing |
| Knowledge Recall | 22.2% | 🟡 Developing |
| Reasoning | 15.7% | 🟠 Early |
| Gold Tier (Discovery) | 16.2% | 🟠 Early |
| Gold Tier (Formulation) | 8.9% | 🔴 Needs work |
Results by Difficulty
| Level | Score | Questions |
|-------|-------|-----------|
| Advanced | 22.9% | 15 |
| Intermediate | 20.0% | 3 |
| Expert | 14.6% | 14 |
Evaluation Notes
- Scoring is strict: 40% keyword matching + 60% reference answer overlap
- Model generates coherent 300-600 word answers — scores understate actual capability
- Best at decision-making (41.4%) — multi-turn reasoning data is effective
- This is a 1B model on 1K samples — a strong baseline, not the ceiling
🏋️ Training
| Parameter | Value |
|-----------|-------|
| Base | MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-heretic |
| Method | LoRA (r=16, α=32, dropout=0.05) |
| Targets | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Epochs | 3 |
| Batch | 2 × 8 (gradient accumulation) |
| LR | 2e-5 (cosine, 50 warmup steps) |
| Precision | fp16 |
| Hardware | 2× NvidiaTeslaT4 (14.6 GB each, Kaggle) |
| Time | 16 min 27 sec |
| Loss | 3.44 → 1.29 (eval, ↓62%) / 2.10 train |
🚀 Quick Start
Python (Transformers)
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"saidutta69/RedPillV1.1",
torch_dtype="auto",
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("saidutta69/RedPillV1.1")
messages = [
{"role": "user", "content": "Design a sustained-release formulation for metformin HCl 500mg using an HPMC matrix system."}
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=1024, do_sample=False)
response = tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)llama.cpp / Ollama
# Pull default quant
ollama run saidutta69/RedPillV1.1
# Or serve with llama.cpp
llama serve -hf saidutta69/RedPillV1.1📂 Files
| File | Size | Description |
|------|------|-------------|
| model.safetensors | 2.0 GB | Merged weights (base + LoRA) |
| tokenizer.json | 9.4 MB | Tokenizer |
| tokenizer_config.json | 565 B | Tokenizer config |
| config.json | 749 B | Model config |
| generation_config.json | 214 B | Generation settings |
| chat_template.jinja | 8.9 KB | Chat template |
📦 Dataset
The RED-PILL dataset that powered this fine-tune:
from datasets import load_dataset
# Full dataset (1,047 instructions)
ds = load_dataset("saidutta69/red-pill-drug-discovery-formulation", data_files="red-pill-full.jsonl")
# Gold tier (17 grade-A instructions)
ds = load_dataset("saidutta69/red-pill-drug-discovery-formulation", data_files="red-pill-full_gold_tier.jsonl")
# Top tier (145 grade A+B instructions)
ds = load_dataset("saidutta69/red-pill-drug-discovery-formulation", data_files="red-pill-full_top_tier.jsonl")🧬 Architecture
MiniCPM5-1B (base)
└─ Heretic abliteration (refusal removal)
└─ LoRA fine-tuning (RED-PILL dataset)
└─ Merged weights → Red Pill V1.1- 1B parameters — runs on gaming PCs, phones, edge devices
- Heretic base — won't refuse pharmaceutical questions
- LoRA fine-tuning — domain knowledge without catastrophic forgetting
- Merged model — standalone weights, no adapter needed
⚠️ Limitations
- 1B parameters — limited reasoning for complex multi-step problems
- 1K training samples — narrow domain; more data = better performance
- English only — no multilingual support
- No real-time data — knowledge follows the base model's cutoff
- Not validated — always verify with domain experts before real-world use
📈 Future Work
| Priority | Task | Impact |
|----------|------|--------|
| 1 | Extended training (10+ epochs) | ↓ loss, ↑ accuracy |
| 2 | More gold-tier data (formulation) | Better formulation answers |
| 3 | Larger dataset (5K+ instructions) | Broader domain coverage |
| 4 | GGUF quantization | Ollama/llama.cpp support |
| 5 | LLM-as-judge evaluation | Fairer scoring |
🙏 Acknowledgments
- Heretic — directional ablation for uncensoring LLMs
- MiniCPM — efficient small language models
- RED-PILL Dataset — 1,047 pharmaceutical instructions
- ChEMBL, DrugBank, PubChem — open pharmaceutical databases
Made with ❤️ by RACER IS OP
Uncensored intelligence for pharmaceutical research