Calibration of measurement units in alignment with a skeleton model to control a computer system

A system having a plurality of sensor modules and a stereo camera and a computing device. Each sensor module has an inertial measurement unit (IMU) measuring its orientation relative to a reference orientation. Different IMUs may have different reference orientations. To calibrate the IMUs with respect to a common reference (e.g., defined based on a standardized pose of a user), the stereo camera captures a stereo image of a respective sensor module attached to a respective portion of the user; the inertial measurement unit of the respective sensor module generates an orientation measurement at a time of capturing the stereo image; and the computing device calculates, based on the stereo image, at least one orientation and uses the orientation and the orientation measurement in determining a rotation that calibrates measurements of the inertial measurement unit relative to the common reference.