Physically Plausible Agent Harness

Code2Worlds-2: Physically Plausible Agent Harness for Embodied World Simulation

¹ School of Computer Science, Peking University ² School of Computing, National University of Singapore

^* Equal contribution. ^† Project lead. ^‡ Corresponding author.

Method

Comparison of the Code2Worlds visual feedback loop and the Code2Worlds-2 simulator feedback loop. — The feedback loops of the two systems. (a) Code2Worlds corrects a scene by rewriting its code from rendered images—a visual loop, which cannot guarantee physical validity. (b) Code2Worlds-2 instead closes the loop through the simulator: Module 1 compiles a SimReady scene, Module 2 critiques the rollout, and simulator feedback guides the revisions to the physical settings and dynamics.

Default convex-hull colliders create phantom support, while SimReady CoACD colliders preserve the chair seat and the space beneath the table. — Schematic comparison of default (top) and SimReady (bottom) scene exports. A single convex hull fills concave spaces, creating phantom support and embedding the box. CoACD colliders preserve these spaces, allowing the sphere to rest on the seat after settling and the box to fit beneath the table. Dashed outlines denote collision geometry; green dots mark contact points.

Overview of the Code2Worlds-2 pipeline and examples from five simulation domains. — Overview of Code2Worlds-2. Two modules are appended to the original Code2Worlds pipeline to produce a simulator-compatible, interactable 4D world, and the illustrated scenes span five domains.

The generation agent emits code and dynamics, receives simulator measurements, and revises physical settings until validation passes. — The Code2Worlds-2 agent harness. The generation agent closes a loop through the simulator: it emits code and a dynamics program, which are compiled, executed, and measured by physical signals; the fused verdict then revises the physical settings until validation passes.

Module 1 compiler stages: collision decomposition, physical attributes, support graph, contact constraints and settling, followed by MJCF export. — The stages of Module 1. Collision geometry is obtained by convex decomposition; density, friction, and restitution are estimated by a vision–language model, while mass and inertia are computed from mesh geometry; a support graph separates supported from floating bodies; and the engine’s contact model, together with a gravity settling pass, produces a simulator-compatible scene exported as MJCF.

Dev15 benchmark charts showing five-domain task composition, executable artifact coverage, penetration measurements below a one-millimeter gate, and normalized physical diagnostics below their failure thresholds. — Dev15 benchmark composition and execution diagnostics. (a) The 15 cases span five domains and three realized task families (N: navigation, D: dynamic interaction, P: pick/transport). (b) All 15 cases have complete executable contracts; 12 additionally retain their authoring scenes. (c) For each domain, bars show the 95th percentile of per-episode penetration P95 values, whereas diamonds show the maximum penetration observed in any episode; both remain below the frozen 1 mm gate, and lower is better. (d) Maximum penetration, Coulomb utilization, generalized-force residual, and quaternion error are normalized by their failure thresholds; lower is better, and all values remain below one.

MuJoCo Simulation

Five environments · three views each

01 /

Indoors

Go2LR Walk

Indoor Scene 02

Vacuum Task

02 /

Industrial

AGV Cargo

AGV Industry

Welding

03 /

Nature Outdoors

Outdoor Scene 01

Outdoor Scene 02

Outdoor Scene 10

04 /

Service

Service Interior

Airport

Hotel

05 /

Urban Outdoors

City AGV

Intersection

Urban Delivery