WEIGHT-SPARSE NPU WITH FINE-GRAINED STRUCTURED SPARSITY

A neural processing unit is provided. It comprises: a weight buffer configured to store weight values in a fine-grain structured sparsity weight arrangement selected from a group of fine-grain structured sparsity weight arrangements comprising at least two arrangements of a 1:4 fine-grain structured sparsity weight arrangement, a 2:4 fine-grain structured sparsity weight arrangement, a 4:8 fine-grain structured sparsity weight arrangement, and a 2:8 fine-grain structured sparsity weight arrangement; a weight multiplexer configured to output one of the weight values stored in the weight buffer as a first operand value based on the selected fine-grain structured sparsity weight arrangement; an activation buffer configured to store activation values; an activation multiplexer coupled to the activation buffer, the activation multiplexer configured to output one of the activation values stored in the activation buffer as a second operand value, the second operand value and the first operand value forming an operand value pair; and a multiplier unit configured to output a product value for the operand value pair.