notesum.ai
Published at December 4The Matrix: Infinite-Horizon World Generation with Real-Time Moving Control
cs.AI
Released Date: December 4, 2024
Authors: Ruili Feng1, Han Zhang1, Zhantao Yang, Jie Xiao1, Zhilei Shu1, Zhiheng Liu1, Andy Zheng2, Yukun Huang3, Yu Liu1, Hongyang Zhang4
Aff.: 1Tongyi Lab; 2University of Waterloo; 3University of Hong Kong; 4Vector Institute

| Feature | Genie | DIAMOND | MarioVGG* | GameNGen* | Oasis* | GameGen-X* | The Matrix |
|---|---|---|---|---|---|---|---|
| Video Length | 2s | Infinite | 6 Frames | Infinite | Infinite | 4s–16s | Infinite |
| Training Corpus | 2D Games | Atari | Mario | DOOM | Minecraft | AAA Games | AAA Games (supervised, small) |
| (unsupervised) | CS:GO | Internet Videos (unsupervised, large) | |||||
| Resolution | 360p | 240p | 720p | 720p | 720p | ||
| Control | Frame-Level | Frame-Level | Video-Level | Frame-Level | Frame-Level | Video-Level | Frame-Level |
| Real-Time | No | Yes | No | Yes | Yes | No | Yes |
| Control Generalization | Yes | No | No | No | No | No | Yes |