World-T3 Test-Time Training as Persistent Memory
for Real-Time Video World Models

Junyi Wu1,2,* Zekun Li1,3,* Chenhongyi Yang1 Chen Guo1 Lingchen Yang1 Yongan Zhang1 Xinyi Zhang1 Hongbing Hu1 Bartek Rymkowski1 Srinath Sridhar3,† Yan Yan2,† Xijie Huang1,†,‡

1Meta Reality Labs 2University of Illinois Chicago 3Brown University

*Equal contribution †Joint last author ‡Project lead

Paper arXiv BibTeX
Revisit consistency: when returning to a previously visited viewpoint, World-T3 recalls the scene faithfully, whereas SANA-WM resynthesizes different content.
Revisit consistency. When the camera returns to a previously visited viewpoint, World-T3 recalls the scene faithfully, whereas autoregressive baselines forget observations beyond their context window and resynthesize new content.
18.31dB
Revisit PSNR
vs. 13.52 for the best baseline
1.08°
Absolute rotation error
vs. 3.34° for the best baseline
20.6FPS
at 1280×704, 1× B200
with live memory updates
O(1)
Memory storage & per-step cost
independent of rollout length

Abstract

Recent advances in video world models have enabled plausible interactive environments, yet long-horizon generation places a fundamental demand on visual memory: the previously observed world must remain consistent as the model continues to generate. Existing autoregressive approaches often implicitly rely on the generative model itself to retain such information, making persistent scene memory difficult to maintain beyond the active context. We introduce World-T3, a camera-controlled video world model with a test-time training memory that consolidates past observations into dedicated model parameters. To robustly update this memory, we propose a recursive least-squares rule that incrementally absorbs novel visual features via online pose-keyed memory updates. Our TTT memory reads out previously observed scene features from the fast-weight state by camera pose, enabling consistent generation with constant storage and per-step computation regardless of rollout length. To facilitate real-time interactive generation with this robust memory, we further distill the autoregressive model into a few-step generator via dual-teacher distribution matching. On a long-horizon revisit benchmark, World-T3 raises revisit PSNR from 13.52 to 18.31 dB and reduces absolute rotation error from 3.344° to 1.075° relative to the strongest baseline on both metrics, while sustaining 20.6 FPS at 1280×704 on a single NVIDIA B200 GPU.

Out-and-Back Revisits

Each 10-second camera trajectory moves forward for 5 s, then retraces its path in reverse. A model with persistent memory should end on the same view it started from. All methods receive the same first frame, text prompt and camera trajectory.

0.0 s
Jump to
Space play / pause   ←→ step a frame   C compare
Text prompt for this scene

Longer Generation with Revisits

0:00 / 0:29

Method

Starting from a pretrained text-image-to-video diffusion model, World-T3 is built in three progressive stages.

Stage 1: Plücker raymaps are encoded and injected into each DiT block via camera-dependent scale and shift.
Stage 1: Camera conditioning. Each frame's camera is encoded as a Plücker raymap, which modulates every DiT block via camera-dependent scale and shift. Attending bidirectionally over the whole clip, this teacher initializes Stage 2 and supplies the reference score for Stage 3.

Results

120 out-and-back clips (10 s, 1280×704, 16 FPS) from the test split. Revisit metrics pair outbound and return frames at the same requested pose, with all gaps beyond our 21-latent-frame context window.

Speed vs. quality

Speed versus revisit PSNR. World-T3 is the only method to the left of the real-time line, and has the highest PSNR.
One B200 GPU. Points left of the dashed line run in real time.

Comparison on the revisit set

Method Steps Revisit ↑ Camera ↓ FPS ↑
Sem. Sim.PSNR RotErr°CamMC
Matrix-Game 3.030.822511.1511.1790.29587.1
HY-World 1.540.856310.8412.8640.34844.5
DreamX-AR40.876811.668.0110.227611.0
LingBot-Fast40.858010.599.8170.26391.6
SANA-WM40.920013.523.3440.120014.2
Ours (Stage 2)500.947616.691.3080.03814.4
Ours (Stage 3)40.961018.311.0750.031820.6

Bold: best, underline: second best. Full table with VBench, RPE and peak memory in the paper.

Qualitative Comparison

Indoor out-and-back comparison across five methods.
Outdoor out-and-back comparison across five methods.
Scene consistency along out-and-back trajectories, indoor (top) and outdoor (bottom). The first and last columns compare the initial and return views.

BibTeX

@article{wu2026worldt3,
  title   = {World-T$^3$: Test-Time Training as Persistent Memory for Real-Time Video World Models},
  author  = {Wu, Junyi and Li, Zekun and Yang, Chenhongyi and Guo, Chen and Yang, Lingchen and
             Zhang, Yongan and Zhang, Xinyi and Hu, Hongbing and Rymkowski, Bartek and
             Sridhar, Srinath and Yan, Yan and Huang, Xijie},
  journal = {arXiv preprint},
  year    = {2026}
}