Methods and systems for predictive engine evaluation, tuning, and replay of engine performance

Disclosed are methods and systems of creating, evaluating, and tuning a predictive engine for machine learning, including steps to deploy the predictive engine with an initial parameter set; receive queries to the deployed engine variant and in response, generate predicted results; receive corresponding actual results; associate the queries, the predicted results, and the actual results with a replay tag; evaluate the performance of the deployed engine variant; generate a new engine parameter set based on tuning of one or more parameters of the initial engine parameter set, according to the evaluation results; deploy the new engine variant to replace the initial engine variant; receive a replay request from an operator specifying the currently or a previously deployed engine variant; and in response to the replay request, replay at least one of the queries, the corresponding predicted results, the actual results, and the evaluation results.