Summary: Embodied AI refers to AI agents that perceive, reason, and act in the physical world — robots, autonomous vehicles, manipulators. The 2024–2025 convergence of world models (planning) + diffusion (generation/control) created the first viable architecture for general-purpose embodied AI.
The Complete Stack
┌─────────────────────────────────────────────────────────────┐
│ EMBODIED AI STACK │
├─────────────────────────────────────────────────────────────┤
│ PLANNING │ WORLD MODEL (dynamics + value) │
│ - Latent MCTS │ - DreamerV3 / Genie / UniSim │
│ - Value-guided diff │ - Diffusion decoder for fidelity │
├─────────────────────────────────────────────────────────────┤
│ CONTROL │ DIFFUSION POLICY (DiT) │
│ - Action chunking │ - Multi-modal action dist │
│ - Receding horizon │ - 10-50 demos → 98% contact tasks │
├─────────────────────────────────────────────────────────────┤
│ PERCEPTION │ VISION ENCODER / VAE │
│ - Image → Latent │ - Frozen encoder for world model │
│ - Proprioception │ - Latent decoder for visualization│
└─────────────────────────────────────────────────────────────┘
Key Architectures (2024–2025)
| System | Planning | Control | Perception | Data |
|---|---|---|---|---|
| DreamerV3 + Diffusion Decoder | Latent dynamics + value | Diffusion decoder | VAE | 100M Atari frames |
| Genie | Latent action model | Controllable video gen | VAE | 1M unlabeled videos |
| UniSim | Trajectory diffusion | Policy distillation | VAE | 1.6M trajectories |
| Diffusion Policy | — (execution) | DiT action diffusion | ResNet/ViT | 10-50 demos |
The Convergence Insight
Diffusion gives world models the generative fidelity they lacked. World models give diffusion the temporal structure and action-conditioning it needed for control.
| Paradigm | Before 2024 | After 2024 |
|---|---|---|
| World Models | Blurry predictions, good planning | Sharp (diffusion decoder), good planning |
| Diffusion Models | Stunning images, no control | Controllable (action cond.), temporal coherence |
This dual gain enables general-purpose embodied AI — one architecture that plans, simulates, and acts.
Decision Guide
| Task | Recommended Architecture |
|---|---|
| Manipulation (low-dim state) | Diffusion Policy (DiT) |
| Visual navigation | DreamerV3 + Diffusion Decoder |
| Sim-to-real transfer | UniSim + domain randomization |
| Learning from video only | Genie (latent action model) |
| Long-horizon (>50 steps) | Mamba/SSM dynamics + value-guided diffusion |
Related Concepts
- world-model — Planning + simulation layer
- diffusion-policy — Low-level control layer
- genie — Video world model
- unisim — Universal simulator
- diffusion-model — Generative foundation
- model-based-rl — Planning framework
- planning-in-latent-space — MPC in latent space
- hrm-reasoning — Hierarchical reasoning for complex tasks
Sources
- world-models-diffusion-ai-robotics — Full convergence analysis
- source-arxiv-2403.12022 — Diffusion Policy
- source-deepmind-genie — Genie (Bruce et al., 2024)
- source-deepmind-unisim — UniSim (Yang et al., 2024)
- source-dreamerv3 — DreamerV3 (Hafner et al., 2023)