METHOD AND APPARATUS FOR TRAINING NEURAL NETWORK, AND DEVICE AND STORAGE MEDIUM

Embodiments of the present invention provide a method and apparatus for training a neural network, and a device and a storage medium. The method comprises: at a first working node among a plurality of working nodes, acquiring a first set of global gradients for a first set of network layers in a neural network, the first set of global gradients being obtained from aggregating local gradients determined by the plurality of working nodes for the first set of network layers in the current training step, and the plurality of working nodes being configured to jointly train the neural network; acquiring a second set of global gradients for a second set of network layers in the neural network, the second set of network layers being different from the first set of network layers, and the second set of global gradients being obtained from aggregating local gradients determined by the plurality of working nodes for the second set of network layers in a training step previous to the current training step; and updating p