METHOD AND APPARATUS FOR UPDATING PARAMETER OF MODEL, DISTRIBUTED TRAINING SYSTEM

The disclosure provides a method for updating a parameter of a model, a distributed training system, and relates to a field of deep learning technologies. The method includes: obtaining a batch training period of batch training data to be trained for a model; increasing priorities of tasks ranked at the bottom in a sequence of gradient communication tasks for parameters of the model when the batch training period is greater than or equal to a preset period threshold; and performing a communication of gradients of the parameters and updating the parameters based on priorities of the gradient communication tasks for the parameters in the model.