DeltaWAM: Delta World Action Models for Bimanual Manipulation

Han Yan^1* Zishang Xiang^1* Haokai Jiang^1* Zeyu Zhang^1*† Qilin Wang¹

Weiyu Guo² Yandong Guo² Boxin Shi¹ Hao Tang^1‡

¹ School of Computer Science, Peking University
² AI² Robotics

* Equal contribution. † Project lead. ‡ Corresponding author.

Method Overview

Four-panel DeltaWAM overview comparing dense visual tokens with sparse visual deltas, Streaming Delta Memory, and evaluation results. — Sparse Visual Deltas & Streaming Delta Memory

DeltaWAM replaces dense future-frame modeling with sparse visual deltas and uses Streaming Delta Memory to update visual context incrementally, improving robustness while reducing training and inference costs.

DeltaWAM architectural variants, the detailed Anchor-Shared network with shared self-attention, and structured training and inference attention masks. — Overview of the DeltaWAM architecture.

(a) Three architectural variants: Anchor-Shared, Separated, and Action-Shared.

(b) Detailed Anchor-Shared architecture, where the Anchor Network and Action DiT retain stream-specific transformations while interacting through layer-wise shared self-attention.

(c) Structured attention masks regulating information flow among anchor, delta, and action tokens.

Real World Evaluation

Task 01

Tabletop Organization

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 01

T1 / 01

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 02

T1 / 02

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 03

T1 / 03

Task 02

Sugar Cube Placement

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 01

T2 / 01

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 02

T2 / 02

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 03

T2 / 03

Task 03

Bowl Stacking

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 01

T3 / 01

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 02

T3 / 02

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 03

T3 / 03

RoboTwin Simulation

Task 01

Adjust Bottle

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 01

T1 / 01

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 02

T1 / 02

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 03

T1 / 03

Task 02

Beat Block with Hammer

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 01

T2 / 01

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 02

T2 / 02

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 03

T2 / 03

Task 03

Rank Blocks by Color

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 01

T3 / 01

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 02

T3 / 02

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 03

T3 / 03

Task 04

Rank Blocks by Size

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 01

T4 / 01

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 02

T4 / 02

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 03

T4 / 03

Task 05

Click Alarm Clock

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 01

T5 / 01

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 02

T5 / 02

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 03

T5 / 03

Task 06

Click Bell

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 01

T6 / 01

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 02

T6 / 02

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 03

T6 / 03

Task 07

Dump Bin into Big Bin

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 01

T7 / 01

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 02

T7 / 02

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 03

T7 / 03

Task 08

Grab Roller

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 01

T8 / 01

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 02

T8 / 02

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 03

T8 / 03

Task 09

Handover Block

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 01

T9 / 01

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 02

T9 / 02

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 03

T9 / 03

Task 10

Handover Microphone

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 01

T10 / 01

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 02

T10 / 02

Eye-to-hand

Left eye-in-hand

Right eye-in-hand

Sample 03

T10 / 03