SYSTEMS AND METHODS FOR ENCODING TEMPORAL INFORMATION FOR VIDEO INSTANCE SEGMENTATION AND OBJECT DETECTION

In a method of encoding of temporal information for stable video instance segmentation and video object detection, a neural network analyzes an input frame of a video to output a prediction template. The prediction template includes either segmentation masks of objects in the input frame or bounding boxes surrounding objects in the input frame. The prediction template is then colour coded by a template generator. The colour coded template, along with a frame subsequent to the input frame, is supplied to a template encoder such that temporal information from the input frame is encoded into the output of the temporal encoder.