SimToolReal: An Object-Centric Policy for Zero-Shot Dexterous Tool Manipulation
An external Stanford study evaluated SimToolReal on a KUKA iiwa 14 arm with a Sharpa Wave hand for zero-shot manipulation of previously unseen tools.
Why it matters
Introduces an object-centric policy for zero-shot dexterous manipulation of unseen tools and tasks.
Evidence notes
- The evaluation covered 120 physical rollouts across 24 tasks, 12 objects and six tool categories.
- The object-centric policy exceeded retargeting and fixed-grasp baselines by 37% while requiring no object- or task-specific training at test time.
Company context
Founded by leadership from Hesai Technology, with operations across Singapore (R&D), Shanghai (manufacturing), and Mountain View (business). Focuses on embodied AI systems combining dexterous manipulation hardware with model-driven control. Develops both robotic platforms and in-house AI models for long-horizon task execution.