Automatic generation of training data for instance segmentation algorithms

Systems and methods to automatically generate training data for machine learning models may include an imaging device to capture imaging data, an image processing or rendering system to receive the imaging data and render a three-dimensional model of an object of interest overlaying the imaging data, an automatic mask extraction or generation system to extract or determine a mask, label, or annotation associated with the three-dimensional model and a plurality of pixels associated with the object of interest from a perspective of the imaging device, and a machine learning model to receive the imaging data and the mask as training data.