Stroke classification for sketch segmentation by fine-tuning a developmental VGGNet16

Sketch segmentation and labeling face two challenges: few samples and few features. 3D data-driven methods use additional labeled 3D meshes to increase samples. However, they are not feasible for the abstract sketches that have no corresponding 3D meshes. And handcrafted feature based methods, although need no 3D meshes, are sensitive to various strokes. To address the challenges, we explore transfer learning based on convolutional neural network (CNN) by fine-tuning a pre-trained CNN to classify strokes for sketch segmentation. We propose a novel informative input for the CNN, making the position information of strokes clear. To improve fine-tuning during transfer learning, we propose to add grouped filter layers to the CNN, making the CNN’s representational capacity incremental. Compared with the state-of-arts, our experimental results achieve 9.7% improvement on the abstract sketch dataset, and 2% improvement on the sketch dataset that has corresponding 3D meshes.