X Square Robot releases X2Real benchmark with 44 manipulation tasks

X Square Robot has published an updated X2Real benchmark for evaluating generalist robot manipulation policies across 44 hierarchical tasks and ten capability dimensions. Its 13 October 2026 technical write-up accompanies a revised research preprint first submitted in September. The benchmark is built on NVIDIA Isaac Lab-Arena and covers fine manipulation, reasoning, memory, tools, bimanual coordination and mobile manipulation.

The authors calibrate visual and physical simulation properties against hardware and report a 0.84 linear correlation between simulated and real-robot evaluation in a comparison spanning eight selected tasks. The technical paper describes separated training and evaluation pipelines, with variations in scene appearance, task conditions and embodiment used to test generalisation.

Supported embodiments include ArtiXon Arm-6A, ARX R5, Franka and the mobile Quanta X1. The demonstration collection contains nearly 300 hours of annotated simulation trajectories, with camera observations, action sequences, language instructions and frame-level subtask labels. A physical domain-specific language supports composition of scenes, subtasks and robots, followed by rollout and evaluation.

The benchmark extends X Square’s work on transferable manipulation models with a shared evaluation environment. Its public task suite ranges from placing objects and opening drawers to coordinated handovers and tool use. Changes in room and table appearance, held-out objects and rules, gripper configurations and camera parameters allow developers to examine where a policy’s performance degrades beyond its training settings.