Tracking user movements to control a skeleton model in a computer system

A system having sensor modules and a computing device. Each sensor module has an inertial measurement unit attached to a portion of a user to generate motion data identifying a sequence of orientations of the portion. The computing device provides the sequences of orientations measured by the sensor modules as input to an artificial neural network, obtains as output from the artificial neural network a predicted orientation measurement of a part of the user, and controls an application by setting an orientation of a rigid part of a skeleton model of the user according to the predicted orientation measurement. The artificial neural network can be trained to predict orientations measured using an optical tracking system based on orientations measured using inertial measurement units and/or to prediction orientation measurements of some rigid parts in a kinematic chain based on orientation measurements of other rigid parts in the kinematic chain.