Learning Representations for Forklift Activity Recognition

Machine Activity Recognition (MAR) is a research topic that focuses on the development of data-driven methods to improve productivity monitoring. The application and the perspective of MAR research jointly influence the diffi- culty of a MAR problem. Unlike previous MAR works, which have studied construction machinery from the viewpoint of the user, this project focuses on logistics equipment from the viewpoint of the original equipment manufac- turer. In terms of the application, forklift trucks have flexible functions and complex usage. The former is an intrinsic characteristic, as forklifts are me- chanically agile, and the latter is an extrinsic factor, as forklift usage can vary greatly with different drivers, loads, work shifts, and warehouse environments. As for the standpoint, manufacturers have customers who use their products all over the world. Studying a single machine or machines in a single site, i.e. the conventional MAR setting, cannot provide a general understanding of the equipment usage. Therefore, existing MAR methods with external sensory data and only supervised learning techniques are impractical in this case. This thesis investigates learning representation-based methods for recog- nizing forklift routine activities using on-board sensory signals. Three methods are developed to capture important data features to overcome the challenges of forklift MAR. First, by pre-training autoencoders with unlabeled data and then fine-tuning them with pseudo-labeled data, discriminative features can be ex- tracted. Classifiers built on these features can outperform conventional MAR solutions that use only the labeled data. Second, training gated recurrent unit networks to recognize activities in different contexts can help to learn a repre- sentation that captures activities and their transitions, which further improves the MAR result. Third, implementing domain adversarial-training neural net- works with pseudo-labeled data can essentially compensate for the limited la- beled data from source domains, leading to representations that are informative for more than one domain. In addition, testing the full method on a real truck has demonstrated the applicability of the proposed method and the feasibility of an online MAR solution.