Optimal Seamline Detection in Dynamic Scenes via Graph Cuts for Image Mosaicking

Li Li, Jian Yao*, Xiaohu Lu, Renping Xie

School of Remote Sensing and Information Engineering, Wuhan University, Wuhan, Hubei, P.R.China

*EMail: jian.yao@whu.edu.cn

*Web: http://www.scholat.com/jianyao http://cvrs.whu.edu.cn

[] The executable files are released.download (15/1/2015).

[] The test dataset is added.download (15/1/2015).

Abstract

In this paper, we present a novel method for creating a seamless mosaic with relatively clean background from two geometrically aligned images captured from a scene with dynamic objects at different time through removing artifacts caused by dynamic objects and geometric misalignments and simultaneously compensating the image regions of dynamic objects based on optimal seamline detection in the graph cuts energy minimization framework. To effectively ensure that all seamlines will be detected in the laterally continuous regions with the high image similarity and the low object dislocation despite the presence of dynamic objects, the energy functions adopted in graph cuts combine the pixel-level similarities of image characteristics, including color and gradient, and the texture complexity. To successfully compensate the image regions covered by dynamic objects for creating a mosaic with relatively clean background, we initially detect them in overlap regions between images based on pixel-level and region-level similarities followed by refinement based on segmentation, then determine their image source in probability based on contour matching, and finally integrate all of these into the energy minimization framework. Experimental results on challenging indoor dynamic scenes demonstrate that our method is capable of generating high-quality mosaics with relatively clean background base on the detected optimal seamlines.

Figure 1: Two geometrically aligned input images A and B in (a) and (b) with a moving object in the overlap region as shown in (c). The image mosaics with different seamlines are presented in (d)-(g), which avoid crossing the moving object. The moving object is visible in (d)-(f), however, it is invisible in (g) by the compensation with the background image regions, which is the goal of our method.

Algorithm

In this paper, not only the color and gradient differences were considered into the cost of each pixel in overlap regions between images but also the texture complexity inspired by HOG (Histogram of Oriented Gradient) was integrated into the cost function. In addition, to create image mosaics with relatively clean background, the dynamic objects are detected initially by computing the pixel-level similarity in the CIELAB color space and the region-level similarity between HOG feature descriptors, then refined by segments generated by the Mean Shift, and finally determined from which image they come in probability based on contour matching. All of these are integrated into the graph cuts energy minimization framework for detecting optimal seamlines concealing the parallax and compensating the image regions covered by dynamic objects for image mosaicking, as shown in Figure 1(g).

Figure 2: An illustration example of the optimal seamline detection method via graph cuts. The thickness of lines between adjacent pixels represents the value of the energy cost and the "cut" denotes the minimum cut, which means the optimal seamline.

Figure 3: The three typical regions (top) happened in indoor scenes and their corresponding normalized texture complexity maps (bottom) where the lighter regions indicate higher texture complexity.

Figure 4: (a)-(b) two geometrically aligned input source images with dynamic objects; (c) the color distance map in CIELAB; (d) the HOG distance map; (e) the final distance map; (f) the dynamic objects binary map refined by the Mean Shift algorithm; (g)-(h) the finally mosaicked images with optimal seamlines without/with dynamic object compensation..