Efficient Batched CPU/GPU Implementation of Orthogonal Matching Pursuit for Python

Finding the most sparse solution to the underdetermined system \(\mathbf{y}=\mathbf{Ax}\), given a tolerance, is known to be NP-hard. A popular way to approximate a sparse solution is by using Greedy Pursuit algorithms, and Orthogonal Matching Pursuit (OMP) is one of the most widely used such solutions. For this paper, we implemented an efficient implementation of OMP that leverages Cholesky inverse properties as well as the power of Graphics Processing Units (GPUs) to deliver up to 200x speedup over the OMP implementation found in Scikit-Learn.