Comparative analysis of deep convolution neural network models on small scale datasets

Deep Learning is a representational model that proves significant impact in computer vision applications by automating the feature extraction process with the advancement and availability of high computing power. Conventional machine learning algorithm works efficiently on large scale data and trained for specific applications that cannot be used for other task. Deep neural networks require large amounts of data, resources, time and high computing devices. Complex deep learning models for a new application domain could be developed using transfer learning with less resources and few labeled training data. An important challenge is negative transfer, which means the initial and target problems are similar. Also, there are no research findings on choosing the right pre-trained model which are closely related to the target dataset and there is no proper measure to analyze the performance. Convolutional Neural Networks models impart significant advancement to the ImageNet visual recognition challenge in the last decade. This work focuses on the use of pre-trained models for small datasets. It analyzes the initialization of parameters, tuning of hyperparameters, parameter optimization, regularization methods, and computation complexity of pre-trained convolutional neural network models. Experiments are performed on three different data sets COREL1k, COIL20 and Garbage images. It shows that the time to re-train the existing pre-trained models was greatly reduced by transfer learning. Transferring weights, instances and layers from models trained in ImageNet dataset could produce better performance in similar small scale datasets, but the performance of dataset which differs in ImageNet dataset was not satisfactory.