Too Much of a Good Thing: When sim2real Efforts Impede Policy Learning (And What to Do About It)
While sim2real efforts are necessary for effective policy transfer to hardware, there is such a thing as too much of a good thing. We argue that sim2real efforts have led to misaligned incentives with policy learning, resulting in simulator lock in and poor policy exploration due to the unreasonable constraints imposed by the real world.
Evidence notes
- Experiments use Apollo 2 Bipedal as the physical evaluation platform.
- The evaluation connects the reported method to Apollo 2 Bipedal hardware in reinforcement_learning tasks.
Company context
Apptronik is a humanoid robotics company focused on general-purpose robots for industrial environments. The company emerged from the University of Texas robotics lab and prior work on NASA's Valkyrie system.