Detection and classification of vehicles using audio visual cues

This paper presents a software-based vehicle detection and classification system capable of classifying traffic into four different classes, namely two-wheeler, three-wheeler, car, and heavy motor vehicle. It uses traffic video collected by a camera mounted on a vehicle parked by the side of a two-lane undivided road. Video frames containing vehicles are identified, both by automatically detecting peaks in Short Time Energy (STE) of the corresponding audio signal and adaptive background subtraction of the video frames, followed by blob subtraction and morphological operations. This may result in multiple images containing the same vehicle, which is eliminated using a Speeded Up Robust Feature (SURF) matching algorithm. Classification of resulting images is attempted in three different ways. In System 1, an SVM trained with explicit features such as Histogram of Gradient (HOG), Local Binary Pattern (LBP), and KAZE are used as a classifier and their performance is compared. In System 2, the task is performed using a deep neural network namely Single Shot Multibox Detector (SSD). The accuracy of the SSD system deteriorates when it is tested using video collected by another camera in a different environment. This issue is addressed in System 3 by retraining the SSD in the new set of images, without the use of manually labeled images. The effectiveness of all the proposed systems is validated using the collected heterogeneous traffic data.