Multiple Regularization and Analysis of Deep Capsule Network

With the increase of layers in deep capsule networks, the overfitting problem also becomes more serious. Capsule-based regularization methods are important to solve this problem. However, little attention has been paid to this field. To fill this gap, we propose five regularization methods from the following aspects. In capsules represented by vectors, two methods are proposed to modify the existence and properties of their activation vectors by disturbing the length and orientation of the vectors. In capsules represented by tensors, capsule-based layer normalization is proposed to improve dynamic routing. In the training strategy, a warm restart learning rate with probability is used to improve the efficiency of training. In reconstruction, a novel image decoder provides a better regularization effect by using multiscale information of images. These regularization methods are investigated on CIFAR10, CIFAR100, and SVHN. Experiments show that using these regularization methods can effectively improve the generalization performance.