Reinforcement Learning for Robotics Part 1: Import a Robot CAD Model into MuJoCo | DigiKey

In this talk

1. 0:15

   What the series will cover

   Hemel outlines a workflow from importing a robot model into MuJoCo to training a balancing agent, deploying it to an ESP32 robot, and adding remote-control commands. Later episodes will address the simulation-to-hardware mismatch through domain randomization and staged training.

2. 2:02

   Why reinforcement learning, and what to know first

   PID control can balance a two-wheeled robot, but Hemel says reinforcement learning can help robots adapt to unfamiliar situations. He points to Boston Dynamics’ Spot and Disney’s BDX droids as examples of robotics work involving learning. He recommends background in probability, neural networks, Python, PyTorch, and reinforcement learning.

3. 7:07

   Measure and export a useful robot model

   The model needs dimensions, component masses, centers of mass, and moments of inertia. Hemel uses a simplified chassis and two smooth wheels, exports STL meshes from FreeCAD, and rotates the model so its axes match MuJoCo’s coordinate convention. He notes that detailed wheel ridges caused contact-simulation problems, so friction can approximate their effect.

4. 27:30

   Build the MuJoCo model and match its timing

   The MJCF file connects the meshes, physical properties, wheel joints, actuators, and sensors, including an accelerometer and gyroscope at the measured IMU location. Hemel sets the simulator step to 5 milliseconds and says the reinforcement learning code and robot firmware must use the same interval. Motor torque and tire friction are estimates that later training can account for through randomization.

5. 43:30

   Run the simulation and try manual control

   Hemel uses a version-pinned Docker setup with MuJoCo and PyTorch on the CPU, then loads the model from a Jupyter notebook. A viewer displays the robot and sensor readings, while keyboard input controls the motors; balancing it manually proves difficult. He ends with a challenge to build a PID controller before the next episode’s reinforcement learning work.