Creating sparsely labeled video annotations

Methods and systems for annotating an object in a video frame. An annotation of an object in a video frame is obtained and a plurality of computerized object trackers are run on the annotated object to automatically label the annotated object in a plurality of subsequent video frames. A computerized user interface aggregating results of the automatic labeling by the plurality of computerized object trackers is presented and, based on the computerized user interface, an identification of a best one of the plurality of computerized object trackers is obtained. The selected computerized object tracker is used to annotate the object in the plurality of subsequent video frames.