Evaluating machine learning model performance by leveraging system failures

A method including monitoring, using a machine learning model, network events of a network. The machine learning model generates fraud scores representing a corresponding probability that a corresponding network event is fraudulent. The method also includes detecting a failure of the machine learning model to generate, within a threshold time, a given fraud score for a given network event. The method also includes determining, by the machine learning model and after the threshold time, the given fraud score. The method also includes logging, responsive to detecting the failure, the given network event in a first table, including logging the given fraud score. The method also includes determining a metric based on comparing the first table to a second table which logs at least the given fraud score and the fraud scores. The method also includes generating an adjusted machine learning model based on the metric.