In order for robots to achieve a wide range of human-level abilities, we will likely need to collect robot action data at a massive scale. However, real-world data collection is challenging to scale up, whereas simulation enables data collection at essentially unlimited levels of scale and diversity. This talk will discuss how we use simulation to train robot humanoid soccer policies that exhibit agility and emergent multi-agent strategic behavior, and to train quadruped locomotion policies that achieve whole body control with language understanding and out-of-distribution generalization. These policies successfully transfer zero-shot to the real world, after minimizing the sim-to-real gap.