METHOD AND APPARATUS FOR PIPELINE PARALLELISM COMPILING

A method for pipeline parallelism compiling is provided, which is executed by one or more processors, and includes receiving a source program associated with training of a machine learning model, determining, based on the source program, a plurality of operation groups including operations executed on training data of the machine learning model, generating a plurality of micro-batches from the training data, and determining, for each of the plurality of micro-batches, a plurality of operation sets corresponding to the plurality of operation groups.