REUSING WEIGHTS AND BIASES IN AN ARTIFICIAL INTELLIGENCE ACCELERATOR FOR A NEURAL NETWORK FOR DIFFERENT MINIBATCH SIZES OF INFERENCES

Provided are a computer program product, system, and method for reusing weights and biases in an artificial intelligence accelerator for a neural network for different minibatch sizes of inferences. A minibatch size is selected of inference jobs batched to process in the accelerator. A representation of a neural network is processed to determine a set of weights and biases for the selected minibatch size to load into the core. The set of weights and biases is loaded into the core for use by the array of processing elements in the core of the accelerator. The weights and the biases are reused in the processing elements for the neural network, loaded for the selected minibatch size, to apply to minibatches of inferences having minibatch sizes less than the selected minibatch size.