Layered motion representation and extraction in monocular still camera videos

A method for segmenting motion data and inferring layering order in a video includes segmenting a first image frame of the video using intra-frame mode estimation, segmenting further image frames of the video using inter-frame mode matching, generating a label change map for a current segmented image frame by generating a binary map of the current segmented image frame, and then marking one or more pixels whose labels have changed with respect to a previous segmented image frame, detecting one or more non-still image segments of the current segmented image frame, detecting one or more moving image segments in the one or more non-still image segments, estimating motion of the one or more moving image segments, representing each moving segment as a vector of seven parameters, and clustering one or more similar moving segment vectors based on corresponding one or more parameters.