SELF-OPTIMIZING LABELING PLATFORM

Systems, methods and products for optimization of a machine learning labeler. In one method, labeling requests are received and corresponding label inferences are generated using a champion model. A portion of the labeling requests and corresponding inferences is selected for use as training data, and labels are generated for the selected requests, thereby producing corresponding augmented results. A first portion of the augmented results are provided as training data to an experiment coordinator, which then trains one or more challenger models using these augmented results. A second portion of the augmented results is provided as evaluation data to a model evaluator, which evaluates the performance of the challenger models and the champion model. If one of the challenger models has higher performance than the champion model, the model evaluator promotes the challenger model to replace the champion model.