HRM-Text-1B Code Fine-tuned
Fine-tuned from HRM-Text-1B on combined code dataset (192M tokens).
Training Details
- Base: HRM-Text-1B (stacked from HTML/CSS 100k checkpoint)
- Dataset: Combined code (Python, JavaScript, TypeScript, SQL, HTML/CSS)
- Tokens: 192M
- Epochs: 3
- Learning rate: 1e-5
Capabilities
- Python code generation
- JavaScript functions
- SQL queries
- General QA (improved over base)
Limitations
- Weak at React/TSX syntax
- HTML/CSS output can be malformed
- TypeScript interfaces not well-formed
Usage
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("InterstellarCG/HRM-Text-1B-Code-FT")
tokenizer = AutoTokenizer.from_pretrained("InterstellarCG/HRM-Text-1B-Code-FT")
Evaluation
| Task | Base | Fine-tuned |
|---|---|---|
| Python (is_prime) | Garbage | Correct |
| JS (reverse array) | Garbage | Correct |
| SQL (join query) | Garbage | Correct |
| QA (Paris capital) | Garbage | Correct |