Background modeling for fixed, mobile, and step- and-stare video camera surveillance

A video camera surveillance system includes a camera for acquiring a video stream and converting terrain space into image space and a method for processing the video stream so that objects in the foreground are detected in the background and threats are identified. In the method, the camera acquires the video stream, images in the video stream are stabilized, an initial background model is constructed, and pixels are classified as either background or foreground for subsequent processing stages such as morphological filtering, image segmentation, and background model updating. In some embodiments of the invention, the camera is fixed in location and attitude, while in other embodiments, the camera is mounted to a moving platform and can alter its attitude.