hep-agent-qwen3.5-9b-mi300x

HEP domain expert — Fine-tuned Qwen/Qwen3.5-9B on High Energy Physics data.

This model is a full fine-tune of Qwen/Qwen3.5-9B on a curated corpus of High Energy Physics literature, experimental data, and synthetic Q&A. Trained on a single AMD MI300X (192 GB HBM3, ROCm 7.0).

Training date: 2026-05-31

Model Overview

Property	Value
Base model	`Qwen/Qwen3.5-9B`
Fine-tuning type	Full fine-tune (NOT LoRA)
Hardware	1× AMD MI300X (192 GB HBM3, ROCm 7.0)
Precision	bfloat16
Context length	2048 tokens
Training data	~50K–100K HEP examples
Optimizer	AdamW 8-bit (bitsandbytes)

Evaluation Results

Benchmark	Score

Intended Use

This model is designed for:

Answering questions about experimental and theoretical particle physics
Explaining detector physics, collision analysis, and data analysis
Solving quantitative physics problems (kinematics, cross-sections, decay calculations)
Summarizing HEP papers and explaining their methodology

Not intended for:

Real-time experimental analysis or ROOT file processing
Safety-critical applications
Medical or regulatory decisions

Training Data

Source	Volume	Description
arXiv hep-ph / hep-ex	~10K papers → Q&A	Theory, phenomenology, experimental
INSPIRE-HEP	~15K records	Paper summaries, detector data
CMS Open Data	~5K examples	Collision analysis, ROOT metadata
PDG (Particle Data Group)	~3K entries	Particle properties, decay modes
Synthetic Q&A	~20K generated	Kinematics, formulas, calculations

Training Configuration

Parameter	Value
learning_rate	8e-06
num_epochs	2
batch_size (effective)	32
sequence_length	4096
optimizer	adamw_8bit

Usage

Basic Generation

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "rajveer43/hep-agent-qwen3.5-9b-mi300x"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

# ChatML format (for Qwen base)
prompt = """<|im_start|>system
You are an expert particle physicist.<|im_end|>
<|im_start|>user
What is the invariant mass of two photons with energies 62.5 GeV each, traveling back-to-back?<|im_end|>
<|im_start|>assistant
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    output = model.generate(**inputs, max_new_tokens=300, do_sample=False)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

vLLM Server (Recommended for Production)

# Install vLLM with ROCm support
pip install vllm --extra-index-url https://download.pytorch.org/whl/rocm7.0

# Launch server
vllm serve rajveer43/hep-agent-qwen3.5-9b-mi300x \
  --dtype bfloat16 \
  --max-model-len 4096 \
  --port 8000

from openai import OpenAI
client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")
response = client.chat.completions.create(
    model="rajveer43/hep-agent-qwen3.5-9b-mi300x",
    messages=[{"role": "user", "content": "Explain the CMS detector architecture."}],
    max_tokens=500,
)
print(response.choices[0].message.content)

Limitations

Knowledge cutoff reflects training data (primarily pre-2025 papers)
May hallucinate specific numerical values; always verify against PDG/PDG Live
Not trained for function-calling or tool-use tasks
Quantitative calculations accurate to ~1-2% for common problems; verify for precision work
Limited coverage of very recent experimental results

Citation

@misc{hep-agent-mi300x-2026,
  title        = {HEP-Agent: Full Fine-Tuning of Qwen/Qwen3.5-9B on High Energy Physics Data},
  author       = {Rathod, Rajveer},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/rajveer43/hep-agent-qwen3.5-9b-mi300x}},
  note         = {Fine-tuned on AMD MI300X (ROCm 7.0) using Unsloth acceleration}
}

License

Apache License 2.0.

Base model weights are subject to their own license: Qwen/Qwen3.5-9B License

rajveer43/hep-agent-qwen3.5-9b-mi300x