Capturing causality and bias in human action recognition

•The existence of biases in biometric datasets can be modeled using causal networks.•We propose a modified temporal convolutional network for human action recognition.•Experiments confirm significant bias in action recognition caused by soft biometrics.•Knowledge distillation approach reduces biases in human action recognition. Human action recognition using various sensors is a mandatory component of autonomous vehicles, humanoid robots, and ambient living environments. A particular interest is the detection and recognition of falls. In this paper, we propose the use of temporal convolution networks guided by knowledge distillation for detecting falls and recognizing types of falls using accelerometer data. Tri-axial accelerometers attached to the body measure the acceleration of the body joints when an action occurs. These data are used for pattern analysis and body action recognition. We demonstrate the existence of biases caused by soft biometrics when recognizing human body actions. We introduce a causal network to capture the influences of biases on system performance and illustrate how knowledge distillation can be applied to mitigate the bias effect.