Region-level motion-based foreground detection with shadow removal using MRFs

This paper presents a new approach to automatic segmentation of foreground objects with shadow removal from an image sequence by integrating techniques of background subtraction and motion-based foreground segmentation. First, a region-based motion segmentation algorithm is proposed to obtain a set of motion-coherence regions and the correspondence among regions at different time instants. Next, we formulate the foreground detection problem as a graph labeling over a region adjacency graph (RAG) based on Markov random fields (MRFs) statistical framework. A background model representing the background scene is built and then is used to model a likelihood energy. Besides the background model, the temporal and spatial coherence are also maintained by modeling it as a prior energy. Finally, a labeling is obtained by maximizing a posterior energy of the MRFs. Experimental results for several video sequences are provided to demonstrate the effectiveness of the proposed approach.