Operating environments: PIM physical terrain research course, OmniH2O physical imitation research station, OmniH2O human-interaction research demonstration, OmniH2O six-task VR collection station
Control modes: Autonomous, Teleoperated
Pim stairs - Research demonstration - Climb stairs with 15-centimetre risers. - environment: PIM physical terrain research course - Hardware: Terrain elevation sensing: Mid-360 LiDAR/FAST-LIO or RealSense T265/D435 configuration - Software: Perceptive Internal Model locomotion policy; Grid elevation map - Control: autonomous - source: arxiv.org
Pim platform gaps - Research demonstration - Jump onto platforms and across gaps in a research course. - environment: PIM physical terrain research course - Hardware: Terrain elevation sensing: Mid-360 LiDAR/FAST-LIO or RealSense T265/D435 configuration - Software: Perceptive Internal Model locomotion policy; Grid elevation map - Control: autonomous - source: arxiv.org
Omni dataset - Research demonstration - Record RGB-D views, head and hand goals, and motor joint targets during VR demonstrations. - environment: OmniH2O six-task VR collection station - Hardware: H1 with fitted wrist motors and Inspire hands; Egocentric camera; OmniH2O configured onboard dual-OrinNX or laptop control setup; VR operator interface - Software: OmniH2O whole-body controller - Control: teleoperated - source: arxiv.org
Omni catch release - Research demonstration - Catch a red box and release it into a trash bin. - environment: OmniH2O physical imitation research station - Hardware: H1 with fitted wrist motors and Inspire hands; Egocentric camera; OmniH2O configured onboard dual-OrinNX or laptop control setup - Software: OmniH2O whole-body controller; Task-trained diffusion imitation policy - Control: autonomous - source: arxiv.org
Omni hammer catch - Research demonstration - Reach into a box and grasp a hammer with the right hand. - environment: OmniH2O physical imitation research station - Hardware: H1 with fitted wrist motors and Inspire hands; Egocentric camera; OmniH2O configured onboard dual-OrinNX or laptop control setup - Software: OmniH2O whole-body controller; Task-trained diffusion imitation policy - Control: autonomous - source: arxiv.org
Omni squat bar - Research demonstration - Squat when an approaching horizontal bar is lower than head height. - environment: OmniH2O physical imitation research station - Hardware: H1 with fitted wrist motors and Inspire hands; Egocentric camera; OmniH2O configured onboard dual-OrinNX or laptop control setup - Software: OmniH2O whole-body controller; Task-trained diffusion imitation policy - Control: autonomous - source: arxiv.org
Omni rock paper scissors - Research demonstration - Respond to a person’s rock-paper-scissors gesture with a winning hand gesture. - environment: OmniH2O physical imitation research station - Hardware: H1 with fitted wrist motors and Inspire hands; Egocentric camera; OmniH2O configured onboard dual-OrinNX or laptop control setup - Software: OmniH2O whole-body controller; Task-trained diffusion imitation policy - Control: autonomous - source: arxiv.org
Omni greeting - Research demonstration - Select and perform greeting gestures in response to a person’s pose. - environment: OmniH2O human-interaction research demonstration - Hardware: H1 with fitted wrist motors and Inspire hands; Egocentric camera; OmniH2O configured onboard dual-OrinNX or laptop control setup - Software: OmniH2O motion primitives; GPT-4o visual motion selector - Control: autonomous - source: arxiv.org