Spatiotemporal modeling covers any task where a model must represent structure that varies across both space (within a single frame or snapshot) and time (across a sequence of frames or snapshots), and where correctness on one axis doesn't imply correctness on the other. Video is the clearest example: a model can produce a spatially plausible single frame while still failing to produce temporally coherent motion across the sequence.
Why it's harder than either axis alone
Full attention over both space and time scales with their product, not their sum — doubling clip length and doubling frame resolution both independently increase the cost of jointly modeling every spatial position at every timestep against every other one. Most efficient spatiotemporal architectures exist specifically to avoid paying that full product cost, typically by factoring space and time apart, or by reasoning over one axis at a coarser resolution than the other — as in a hierarchical decomposition where the temporal/structural plan is coarse and per-frame detail is fine.