Stage 1 transfers an arm-centric WAM to the quadruped and accelerates it with OPSD; Stage 2 improves it per task and merges the teachers with M-OPD.
Stage 1
Cross-Morphology Policy Adaptation and Acceleration
Stage 1.1: Cross-Morphology Policy Transfer
1Arm-centric WAM
Source embodiment
Task instruction
Text Encoderℓ
Image observation
Video Streamzk → Zk
Future imagination Autoregressive
Action StreamAk
Robot action
2Interface Adaptation
Visual Adaptation
Head
Left wrist
Right wrist
Ego-centric
Top-down
Action Adaptation
Joint + Gripper(30D)
Body + Foot End(9D)
Controller Execution
Low-level Whole-Body Controller 𝒞
Joint torque
3QLM-Bench Data Synthesis
Task prompts
Press the button.
Push the box inside the outlined area.
Kick the ball through the goal.
…
Push the oven door until the door closes.
LLMs
Reference Trajectory
Scene Randomization
Automatic Synthesis
Demonstrations
press button
push box
kick ball to goal
close oven door
4Stage 1.2: On-Policy Self-Distillation (OPSD)
High-NFE Quadrupedal WAM πT
On-Policy Self-Distillation (OPSD)
Low-NFE Quadrupedal WAM πS
Generation time / chunk
High-NFE2.226 s
Low-NFE0.159 s
14.0× Speed-up
5Closed-loop Inference
“Press the button.”
Real State
Inverse Dynamics
Future Imagination
EndStart
every chunk
Stage 1: Cross-Morphology Policy Adaptation and Acceleration. Stage 1.1 fine-tunes an arm-centric WAM on quadruped demonstrations synthesized in QLM-Bench. Visual Adaptation replaces the head and wrist views (images from Mobile ALOHA) by the ego-centric and top-down views, Action Adaptation replaces the 30-dimensional arm command by the 9-dimensional quadruped command, and the whole-body controller 𝒞 converts each command into joint torques. Stage 1.2, OPSD, distills the transferred policy, as the high-NFE teacher πT, into the 1/2-NFE student πS; the generation time counts the sampling of video and action only. Bottom right shows the closed loop of the policy on the task “Press the button.” Imagined views and rendered demonstrations are illustrations.
On-Policy Self-Distillation (Stage 1.2). Left (Training Context): training contexts mix the teacher cache, which the frozen teacher collects in closed loop before training, with the contexts that the student reaches in its own streaming rollouts (H ∼ d). Right (Target & Loss): on each context, teacher and student sample from the same noise, and the student’s one-step video stream and two-step action stream are regressed to the targets of the teacher’s high-NFE sampler, with a stop-gradient on the plan that conditions the student’s action. Plan images and rendered actions are illustrations, not model outputs.
Stage 1.1 · fine-tuning
Video loss
Action loss
Training (per update)Training (100-update mean)Validation
Stage 1.2 · OPSD
OPSD: video
OPSD: action
Teacher cache only (η = 1)Streaming only (η = 0)OPSD (Ours, η = 0.5)
Left: fine-tuning losses of the transferred policy (Stage 1.1), on a logarithmic scale. Light: training loss of each update; dark: its centered moving average over 100 updates, i.e. one pass over the 800 training demonstrations; dashed: validation loss on 10 held-out demonstrations per task, every 100 updates. Right: distillation loss of OPSD. The held-out loss is evaluated every 50 updates. Held-out loss of the students of the three training-context sources, on the same 64 held-out contexts, with a square-root horizontal axis; the dotted line marks 300 updates, after which the students are evaluated. The held-out contexts are drawn from the teacher cache, so the curves do not rank the sources: they disadvantage Streaming only, which trains on no cached contexts.
Stage 2
Task-Specific Improvement and Merging
1Stage 2.1: Task-Specific Reinforcement Learning
OPSD teacherHigh-NFE
Candidate Windows
Task-specific Reward
DiffusionNFT Update
Specialist teacher(push box)
Candidate Windows
Task-specific Reward
DiffusionNFT Update
Specialist teacher(open door)
2Stage 2.2: Multi-Teacher On-Policy Distillation
Task Router
M-OPD
M-OPD student 1/2-NFE
OPSD teacherHigh-NFE
OPSD studentInit.
Task-Specific Improvement and Merging (Stage 2). Stage 2.1 tunes the OPSD teacher by reinforcement learning into a specialist teacher for each selected task, and Stage 2.2, M-OPD, distills the teachers, routed by task, into one 1/2-NFE student that starts from the OPSD student.
Left: specialist teachers during Stage 2.1. Solid line (left axis): reward gain over the OPSD teacher on held-out start states after each round, paired by start state and noise, with a band of ±1 standard error over scenes; the two rewards measure different quantities (the box reward in mm, the offset of the foot on the door handle in cm). Squares (right axis): closed-loop success over 32 episodes after rounds 4, 9, and 14; the dotted line is the success of the OPSD teacher. For opening the door, the success falls from 19 of 32 episodes after round 4 to 6 after round 9 while the reward gain stays positive, so the reward is only a proxy of success there; the specialist teacher is the update after round 4. Right: distillation loss of M-OPD. The held-out loss is evaluated every 50 updates. The M-OPD student starts from the OPSD student and learns from the specialist teachers routed by task. The light line is the training loss of every update, computed on the contexts sampled for that update rather than on the held-out contexts; the dark line is its centered moving average over 25 updates, with shorter windows at both ends; the circles are the held-out loss. The held-out contexts of M-OPD come from its own teacher cache, so its loss is not on the scale of the OPSD losses.
QLM-Bench
The ten tasks of QLM-Bench, one demonstration per task from the observer camera of the dataset.
Close a fridge door“Push against the fridge door until the door closes.”Close an oven door“Push the oven door until the door closes.”Turn a valve“Rotate the valve lever upward to the upright position.”Pull open a drawer“Slide your foot into the drawer handle opening and pull the drawer outward.”Open a room door“Press down the lever handle, then push the door open.”Press a button“Press the button.”Lift a basket“Lift the basket by its handle.”Kick a ball through a goal“Kick the ball through the goal.”Push a box into a target area“Push the box inside the outlined area.”Sweep a block into a target area“Sweep the block into the outlined area.”