HARDWARE ARCHITECTURE FOR PROCESSING TENSORS WITH ACTIVATION SPARSITY

A hardware accelerator that is efficient at performing computations related to tensors. The hardware accelerator may store a complementary dense process tensor that is combined from a plurality of sparse process tensors. The plurality of sparse process tensors have non-overlapping locations of active values. The hardware accelerator may perform elementwise operations between the complementary dense process tensor and an activation tensor to generate a product tensor. The hardware accelerator may re-arrange the product tensor based on a permutation logic to separate the products into groups. Each group corresponds to one of the sparse process tensors. Each group may be accumulated separately to generate a plurality of output values. The output values may be selected in an activation selection. The activation selection may be a dense activation or a sparse activation such as k winner activation that set non-winners to zeros.