A Unified Framework for Street-View Panorama Stitching

Li Li, Jian Yao*, Renping Xie, and Menghan Xia

School of Remote Sensing and Information Engineering, Wuhan University, Wuhan, Hubei, P.R.China

*EMail: jian.yao@whu.edu.cn

*Web: https://cvrs.whu.edu.cn http://www.scholat.com/jianyao

Wei Zhang

School of Control Science and Engineering, Shandong University, Jinan, Shandong, P.R.China

Abstract

In this paper, we propose a unified framework to generate a pleasant and high-quality streetview panorama by stitching multiple panoramic images captured from the cameras mounted on the mobile platform. Our proposed framework is comprised of four major steps: image warping, color correction, optimal seamline detection and image blending. Since the input images are captured without a precisely common projection center from the scenes with the depth differences with respective to cameras to different extents, such these images cannot be precisely aligned in geometry. So, an efficient image warping method based on the dense optical flow field is proposed to greatly suppress the influence of large geometric misalignment at first. Then, to lessen the influence of photometric inconsistencies caused by the illumination variations and different exposure settings, we propose an efficient color correction algorithm via matching extreme points of histograms to greatly decrease color differences between warped images. After that, the optimal seamlines between adjacent input images are detected via the graph cuts energy minimization framework. At last, the Laplacian pyramid blending algorithm is applied to further eliminate the stitching artifacts along the optimal seamlines. Experimental results on a large set of challenging street-view panoramic images captured form the real world illustrate that the proposed system is capable of creating high-quality panoramas.

Figure 1: Our proposed unified framework for the street-view panorama stitching system.

Experimental Results

The images of outdoor scenes captured by an integrated multi-camera equipment with 6 Nikon D7100 cameras of 24 million pixels installed on a mobile vehicle platform to illustrate the performance of our method for mosaicking panoramic images. Figures 4 show the seamline detection results of panoramic images.

Figure 2: The pictures in the left column are the results of optimal seamline detection (The results of the second paper in References), and we can find that there are many artifacts caused by large geometric misalignments and color differences. The pictures in the middle column is the results of Xiong and Pulli's approach(The third paper in the references). Due to this approach has not eliminate the large geometric misalignments before image mosaicking, so we use the warped images generated by our proposed image warping algorithm as the input images. The pictures in the right column are the corresponding results of last panoramas generated by our proposed framework, and those large geometric misalignments and color differences are eliminated. Please left-click the pictures to enlarge them.

References

  • Li Li, Jian Yao*, Renping Xie, Menghan Xia. A unified framework for street-view panorama stitching, submitted to Remote Sensing, Sept. 2016.   [PDF]
  • Li Li, Jian Yao*, Xiaohu Lu, Jinge Tu, Jie Shan. Optimal seamline detection for multiple image mosaicking via graph cuts, ISPRS Journal of Photogrammetry and Remote Sensing, 113(2016):1-16, 2016.   [PDF]
  • Yingen Xiong and Kari Pulli. Fast panorama stitching for high-quality panoramic images on mobile phones. IEEE Transactions on Consumer Electronics, 56(2):298-306, 2010.   [PDF]