A reconstructed feasible solution-based safe feature elimination rule for expediting multi-task lasso

Multi-task lasso (MTL) is an effective algorithm to handle multi-task problems. By introducing ℓ2,1-norm, it can realize joint feature selection across a group of related tasks. But it is time-consuming when handling high-dimensional problems. Motivated by the row sparsity of the optimal solution, an improved safe feature elimination rule termed IEDPP is proposed to accelerate the training process. It could delete most of the redundant features before we solve the problem. Then the computational efficiency will be improved since only a reduced problem should be solved. Moreover, the properties of the projection operator and the reconstructed feasible solutions ensure the safety of the proposed method. That is to say, it will derive an identical optimal solution to the original problem both in theory and in practice. But our IEDPP could only be used once before solving, there are still some redundant features that are not deleted. Therefore, we further propose an integrated IIEG-ML rule by combining IEDPP with GAP. Then, more and more redundant features could be deleted as the algorithm converges. Moreover, by embedding IIEG-ML into the grid search method, the whole training process will be accelerated. Finally, an improved Nesterov's method is used to solve the reduced problems. Experimental results on different datasets confirm the effectiveness of our method.