SCENE-AGNOSTIC REGRESSION ON PIXEL-LEVEL ANNOTATIONS

A computer-implemented method of training a machine learning model for regression on pixel-level annotations in images is disclosed. The method comprises pre-training an image encoder and a decoder for cross-view completion, constructing training tuples, each comprising a first image, associated with dense pixel-level annotations, and one or more second images, each associated with sparse pixel-level annotations. The image encoder generates first image tokens and second image tokens from the first and second images. A feature mixer generates sets of augmented second image tokens by augmenting the second image tokens with the associated sparse pixel-level annotations. The method further comprises processing, by the decoder, the first image tokens and the augmented second image tokens to generate prediction data for the first image, the prediction data comprising dense pixel-level predictions, and fine-tuning the machine learning model based on the prediction data and the dense pixel-level annotations.