Summary: Diffusion Policy (Chi et al., RSS 2024) introduces diffusion models as robot policies — learning action distributions via denoising conditioned on observations. Achieves 98% success on contact-rich manipulation with 10-50 demonstrations, outperforming VAE/GMM policies. Uses DiT backbone (~100M params) with action chunking.

Core Idea

Instead of predicting a single action (like BC) or a distribution mean (like Gaussian policy), diffuse the entire action sequence:

a_{0:H} ← denoise(epsilon, obs_t)
  • Conditioning: Observation history (images, proprioception)
  • Output: Action chunk a_{t:t+H} (e.g., 16 steps)
  • Architecture: DiT (Diffusion Transformer) ~100M params

Why Diffusion for Actions?

Property BC / Gaussian Policy Diffusion Policy
Multi-modality ❌ Mean of modes = invalid action ✅ Naturally models "grasp left OR right"
Contact-rich tasks ❌ Struggles with discontinuities ✅ 98% success (vs ~60-70% BC)
Demos needed 100-1000+ 10-50
Sim-to-real Brittle Robust (denoising = correction)

Architecture

Observation (img + proprio) → Encoder → Conditioning
                                  ↓
Action chunk (H steps) → DiT (Denoiser) → Clean action chunk
                                  ↑
                            Timestep t + noise

Key design choices:

  • Action chunking — Predict H steps, execute first k, re-plan (receding horizon)
  • DiT backbone — Transformer scales better than U-Net for conditional diffusion
  • Noise schedule — Cosine or linear, tuned for action space

Workflow Diagram

Diagram: (Mermaid diagram - view source for diagram code)

flowchart LR
    subgraph TRAINING [Training Process]
        OBS_T[Observation] --> ENC_T[Encoder]
        ENC_T --> COND_T[Conditioning]
        NOISE_T[Noise Schedule t] --> COND_T
        NOISY_ACT_T[Noisy Action Chunk] --> DIT_T[DiT Denoiser]
        COND_T --> DIT_T
        DIT_T --> CLEAN_ACT_T[Clean Action Chunk]
        CLEAN_ACT_T --> LOSS[Loss: MSE vs GT Action]
        GT_ACT_T[Ground Truth Action] --> LOSS
    end

subgraph INFERENCE [Inference Process]
    OBS_I[Observation] --> ENC_I[Encoder]
    ENC_I --> COND_I[Conditioning]
    NOISE_I[Noise Schedule t] --> COND_I
    RANDOM[Random Noise] --> NOISY_ACT_I[Noisy Action Chunk]
    NOISY_ACT_I --> DIT_I[DiT Denoiser]
    COND_I --> DIT_I
    DIT_I --> PRED_ACT[Predicted Action Chunk]
    PRED_ACT --> EXEC[Execute First k Steps]
    EXEC --> REPLAN[Re-plan: New Observation]
    REPLAN --> OBS_I
end

style TRAINING fill:#f9f9f9,stroke:#ccc
style INFERENCE fill:#f0f8ff,stroke:#8db6cd

Action Chunking:

  • Predict horizon H (e.g., 16 steps)
  • Execute first k steps (e.g., 8 steps)
  • Replan with new observation
  • Provides reactive control while maintaining foresight

Results (from paper)

Task Demos Diff. Policy VAE Policy GMM Policy BC
Can (sim) 50 98% 72% 61% 58%
Threading (sim) 50 94% 45% 38% 32%
Coffee (sim) 50 91% 52% 41% 38%
Real robot (can) 50 sim + 10 real 90% — — —

Key finding: Multi-modality handling = main advantage. VAE/GMM collapse modes on contact tasks.

Limitations (from paper)

Limitation Severity
Chronic planning horizon (fixed chunk H) Medium
No explicit long-horizon reasoning Medium
Requires action labels (not video-only) Low (by design)
10 denoising steps = latency Low (can distill)

For Your Research

Relevance Application
Diffusion Policy concept Base for all diffusion robotics
DiT for robotics Architecture reference for your world model
Action chunking Pattern for receding-horizon control
Multi-modality Key argument for diffusion > VAE/GMM

In World Model Stack

Layer Role
World Model (Dreamer/Genie/UniSim) Simulate dynamics, plan latents
Diffusion Policy Execute low-level actions from plan
Value Function Guide planning (value-guided diffusion)

DreamerV3 + Diffusion Decoder → Sample-efficient planning + sharp execution.

Related Concepts

Sources

  • Primary: arxiv-2403.12022 — Chi et al., "Diffusion Policy: Visuomotor Policy Learning via Action Diffusion" (RSS 2024) — arXiv:2403.12022