Hierarchical Reasoning Model (HRM)

Summary: HRM (Sapient Intelligence, 2025) is a brain-inspired recurrent architecture with two coupled modules operating at different timescales: a slow High-level (H) module for abstract planning and a fast Low-level (L) module for detailed computation. It achieves deep effective depth via hierarchical convergence without BPTT, using a 1-step gradient approximation grounded in Deep Equilibrium Models. With only 27M params and ~1K training samples (no pre-training, no CoT), HRM reaches 40.3% on ARC-AGI-1, near-perfect on Sudoku-Extreme and Maze-Hard — surpassing much larger CoT models.

Core Architecture

Two-Module Hierarchy (Fig. 1, §2)

Input x → Embedding f_I → x̃
                    ↓
         ┌────────────┴────────────┐
         │   N cycles × T steps    │
         │   (e.g., N=2, T=2)      │
         └────────────┬────────────┘
                      ↓
    z_L^i = f_L(z_L^{i-1}, z_H^{i-1}, x̃)   # Low-level: every step
    z_H^i = f_H(z_H^{i-1}, z_L^{i-1})       # High-level: every T steps (i ≡ 0 mod T)
                      ↓
              Output f_O(z_H^{NT})
  • Total forward steps: N × T (e.g., 4 steps per segment)
  • H-module: Updates once per cycle (slow, abstract)
  • L-module: Updates every step (fast, detailed)
  • Coupling: L receives H's state; H receives L's final state per cycle

Module Details (§2, §3.4)

Module Architecture Role
f_I Embedding layer Input tokens → vectors
f_L 4-layer Transformer encoder Fast, detailed computation
f_H 4-layer Transformer encoder (same dims) Slow, abstract planning
f_O Linear + softmax (stablemax for small data) State → token probs

Both recurrent modules: Post-Norm, RMSNorm, RoPE, SwiGLU, no bias, truncated LeCun Normal init.

Hierarchical Convergence (§2, Fig. 3)

Key insight: Standard RNNs converge too fast (premature convergence), capping effective depth. HRM avoids this via hierarchical convergence:

  1. L-module converges to local equilibrium within each cycle (fixed z_H)
  2. H-module updates using converged z_L → new context for L
  3. L-module resets (implicit via new H context) → starts new convergence toward different equilibrium
  4. Result: Sequence of distinct, stable computations → effective depth = N × T

Fig. 3 (arxiv-2506.21734): H-module residuals steadily decrease; L-module shows repeated convergence spikes (reset each cycle). Standard RNN: residuals → 0 quickly. Deep NN: vanishing gradients.

1-Step Gradient Approximation (§2, Fig. 4)

Problem: BPTT Memory

  • BPTT needs O(T) memory for T timesteps
  • Biologically implausible (no full history replay)

Solution: Implicit Function Theorem + 1-Step Approximation

Assume L-module reaches fixed point z_L^* = f_L(z_L^*, z_H, tilde{x}) per cycle. Then H-update: z_H^k = f_H(z_H^{k-1}, z_L^*) = F(z_H^{k-1}). Fixed point: z_H^* = F(z_H^*).

Exact gradient (IFT):

(∂ z_H^*)/(∂ theta) = (I - J_F|_{z_H^*})^{-1} (∂ F)/(∂ theta)|_{z_H^*}

1-Step Approximation (Neumann series truncation to I):

(∂ z_H^*)/(∂ theta_H){∂ theta_H} ≈ (∂ f_H)/(∂ theta_H), quad (∂ z_H^*)/(∂ theta_L) ≈ (∂ f_H)/(∂ z_L^*) · (∂ f_L)/(∂ theta_L)

Gradient path: Output head → final H-state → final L-state → input embedding Memory: O(1) vs BPTT O(T) — no unrolling!

Implementation (Fig. 4 pseudocode)

# Last 2 steps WITH gradients; previous steps detached
z_L = L_net(z_L, z_H, x)  # with grad
z_H = H_net(z_H, z_L)     # with grad
loss = cross_entropy(output_head(z_H), y)
loss.backward()  # Only flows through last L and H steps

Deep Supervision (§2.4, Fig. 4)

Inspiration: Brain theta oscillations gate learning phases.

Instead of single forward pass, run multiple segments (up to N_{sup}=16):

  1. Forward pass → state z^m, prediction hat{y}^m
  2. Compute loss Loss(hat{y}^m, y)
  3. Update params immediately
  4. Detach z^m (stop gradient) → use as init for next segment
  5. Repeat

Effect:

  • Frequent feedback to H-module
  • Regularization (like deep supervision in DEQs)
  • Empirically superior to Jacobian-based regularization

Adaptive Computation Time (ACT) (§2.4, Fig. 5)

Q-Learning Halting Policy

  • Q-head on z_H: hat{Q}^m = Sigma(theta_Q^T z_H^{mNT}) → halt/continue values
  • MDP: State=z^m, Actions={halt, continue}
  • Rewards: halt → mathbf{1}\{hat{y}^m = y\}; continue → 0
  • Q-targets (Eq. 185-198):
    • hat{G}_halt^m = mathbf{1}\{hat{y}^m = y\}
    • hat{G}_continue^m = max(hat{Q}_halt^{m+1}, hat{Q}_continue^{m+1}) (or hat{Q}_halt if m ≥ N_{max})

Stochastic Halting

  • M_{min}: with prob varepsilon sample from \{2..M_{max}\}, else 1 (encourages longer thinking)
  • Halt if: m > M_{max} OR (hat{Q}_halt > hat{Q}_continue AND m ≥ M_{min})

Results (Fig. 5)

  • ACT adapts compute to difficulty: low avg steps, stable as M_{max} grows
  • Inference-time scaling: Train with M_{max}=8 → test with M_{max}=16 → accuracy improves
  • Sudoku: strong scaling; ARC: minimal gains (few transformations needed)

Brain Correspondence (§4, Fig. 8)

Dimensionality Hierarchy

  • Participation Ratio (PR): (sum lambda_i)^2 / sum lambda_i^2 (effective dimensionality)
  • Mouse cortex: PR increases up hierarchy (Spearman rho=0.79, p=0.0003)
  • Trained HRM (Sudoku):
    • z_H PR = 89.95 (high-dim)
    • z_L PR = 30.22 (low-dim)
    • Ratio z_H/z_L ≈ 2.98 ≈ mouse cortex ≈ 2.25
  • Untrained HRM: No hierarchy (both low, ~40)

Conclusion: Hierarchical dimensionality is an emergent property of training, not architecture.

Benchmarks & Results (Fig. 1, §3)

Task HRM (27M, 1K) o3-mini-high Claude 3.7 (8K) Direct Pred (8L)
ARC-AGI-1 40.3% 34.5% 21.2% ~19%
Sudoku-Extreme ~100% 0% (CoT fails) 0% 16.9% (full data)
Maze-Hard 30×30 ~100% — — <20% (175M, 1M samples)

ARC details: 1000 augmented variants per test input, majority vote over 2 top predictions.

Ablation Insights (from arxiv-2510.04871 §2.4)

Component ARC-AGI-1 Accuracy Δ
Single forward pass (no deep supervision) 35.7% baseline
+ Deep supervision (single module) 39.0% +3.3%
+ Hierarchical recursion (H+L) 39.0% +0%
Full HRM (deep supervision + hierarchy) 40.3% +4.6%

Independent finding: Deep supervision is the primary driver; recursive hierarchy adds marginal benefit.

Training Details (§3.2)

  • Optimizer: Adam-atan2 (scale-invariant Adam) + constant LR + linear warmup
  • Loss: Seq2seq cross-entropy (avg over tokens)
  • Init: Truncated normal (std=1, trunc=2) for z^0; fixed during training
  • Data: ~1000 samples per task (ARC train + augmentations; Sudoku/Maze 1000 each)
  • Augmentation: ARC — translations, rotations, flips, color perms; Sudoku — band/digit perms

Open Questions & Limitations (§6)

Question Status
Turing completeness Theoretically yes (with memory/time), like Universal Transformer
RL vs gradient supervision HRM uses dense gradients; RL unlocks existing CoT, not new mechanisms
Linear attention integration HRM uses full attention; hierarchical memory could be future work
Causal necessity of dim hierarchy Correlational; intervention needed
Generalization to language Tested on puzzles; LLM integration unexplored

Key Claims with Sources

Claim Source Locator Confidence
Two recurrent modules (H slow, L fast) with hierarchical convergence arxiv-2506.21734 §2, Fig. 3 0.95
1-step gradient approx via IFT + Neumann truncation arxiv-2506.21734 §2, Eq. 120-142 0.95
Deep supervision with detached segments (up to 16) arxiv-2506.21734 §2.4, Fig. 4 0.95
ACT via Q-learning with stochastic M_{min} arxiv-2506.21734 §2.4, Eq. 175-198 0.95
27M params, 1K samples, no pre-training/CoT arxiv-2506.21734 Abstract, §3.2 0.95
40.3% ARC-AGI-1, ~100% Sudoku-Extreme, Maze-Hard arxiv-2506.21734 Fig. 1, §3.2 0.95
Brain PR hierarchy: mouse cortex rho=0.79; HRM z_H/z_L ≈ 2.98 arxiv-2506.21734 §4, Fig. 8 0.9
Untrained HRM shows no PR hierarchy (control) arxiv-2506.21734 §4, Fig. 8(e,f) 0.95
Deep supervision primary driver (ablation) arxiv-2510.04871 §2.4 0.9

Related Concepts

Sources

  • arxiv-2506.21734: "Hierarchical Reasoning Model" (Wang et al., 2025)
  • arxiv-2510.04871: "Less is More: Recursive Reasoning with Tiny Networks" (Jolicoeur-Martineau, 2025) — ablation analysis