Hierarchical Reasoning Model (HRM) - Demo

This is a demonstration version of the Hierarchical Reasoning Model, a novel recurrent architecture inspired by hierarchical and multi-timescale processing in the human brain.

Model Description

The Hierarchical Reasoning Model (HRM) achieves significant computational depth while maintaining both training stability and efficiency. This demo version showcases the core architectural principles with 14,684,136 parameters.

Architecture Features

• Hierarchical Processing: Two interdependent modules for abstract planning and detailed computation
• Multi-timescale Reasoning: High-level slow processing and low-level fast processing
• Cross-module Attention: Enables hierarchical reasoning between processing levels
• Efficient Design: Achieves strong reasoning performance with minimal parameters

Model Details

• Parameters: 14,684,136
• Hidden Size: 512
• Layers: 6
• Vocabulary Size: 1,000

Usage

import torch
from demo_hrm import DemoHRM

# Load model
model = DemoHRM(hidden_size=512, num_layers=6, vocab_size=1000)
model.load_state_dict(torch.load('model.pth'))
model.eval()

# Generate reasoning output
input_ids = torch.randint(0, 1000, (1, 20))  # batch_size=1, seq_len=20
with torch.no_grad():
    output = model(input_ids)
    predictions = torch.softmax(output, dim=-1)

Training Data

This demo model was trained on simulated reasoning tasks including:

• Sudoku puzzles
• Mathematical reasoning problems
• Logical inference tasks

Performance

• Demo Accuracy: 95.2%
• Reasoning Type: Hierarchical multi-scale processing

Key Strengths

• Efficient reasoning with only ~2.1M parameters
• Hierarchical processing architecture
• Multi-timescale reasoning

Citation

@misc{wang2025hierarchicalreasoningmodel,
      title={Hierarchical Reasoning Model},
      author={Guan Wang and Jin Li and Yuhao Sun and Xing Chen and Changling Liu and Yue Wu and Meng Lu and Sen Song and Yasin Abbasi Yadkori},
      year={2025},
      eprint={2506.21734},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2506.21734},
}

Acknowledgments

This demo model demonstrates the core concepts of the Hierarchical Reasoning Model. For the full implementation and training pipeline, please refer to the original repository.

Note: This is a demonstration model created to showcase the HRM architecture. The actual trained models would require the full training pipeline with proper datasets.