Energy-based model of least squares twin Support Vector Machines for human action recognition

Human action recognition is an active field of research in pattern recognition and computer vision. For this purpose, several approaches based on bag-of-word features and support vector machine (SVM) classifiers have been proposed. Multi-category classifications of human actions are usually performed by solving many one-versus-rest binary SVM classification tasks. However, it leads to the class imbalance problem. Furthermore, because of environmental problems and intrinsic noise of spatio-temporal features, videos of similar actions may suffer from huge intra-class variations. In this paper, we address these problems by introducing the Energy-based Least Square Twin Support Vector Machine (ELS-TSVM) algorithm. ELS-TSVM is an extended LS-TSVM classifier that performs classification by using two nonparallel hyperplanes instead of a single hyperplane, as used in the conventional SVM. ELS-TSVM not only could consider the different energy for each class but also it handles unbalanced datasets׳ problem. We investigate the performance of the proposed methods on Weizmann, KTH, Hollywood, and ten UCI datasets which have been extensively studied by research groups. Experimental results show the effectiveness and validity of noise handling in human action and UCI datasets. ELS-TSVM has also obtained superior accuracy compared with the related methods while its time complexity is remarkably lower than SVM. •We have extended the LS-TSVM algorithm to the energy based model called ELS-TSVM for human action recognition.•The energy for each hyperplane has been introduced to be flexible in the face of outliers of each action.•ELS-TSVM performed several orders of magnitude faster than SVM.•SVM leads to the unbalance dataset problem in multi-class classification. But, ELS-TSVM addresses it.