Color Correction and Image Blending for Panorama Stitching via Extreme Point Matching of Luminance Histogram


Yaping Liu, Jian Yao*, Mi Zhang, and Li Li

School of Remote Sensing and Information Engineering, Wuhan University, Wuhan, Hubei, P.R.China

*EMail: jian.yao@whu.edu.cn

*Web: http://www.scholat.com/jianyao http://cvrs.whu.edu.cn


1. Abstract

In this paper, we propose a novel image blending method for panorama stitching from multiple images based on the extreme point matching of the luminance histogram. To reduce the bright differences between images, an automatic adjustment of contrast is firstly applied on individual images based on their own color histograms. Both the Probability Density Function (PDF) and the Cumulative Distribution Function (CDF) with respect to luminance histograms are secondly utilized to obtain extreme points of the luminance histograms that are capable of best illustrating statistical characteristics of the image, which are robustly matched according to two predefined rules to construct the relationship between two involved images. Thirdly a pixel-wise color correction is employed in the whole image and a simple alpha transition correction strategy is used to ensure color continuity in the overlapped regions between images. Finally a multi-band blending with optimal seamlines is adopted to create a seamless panorama. Experimental results show that our method is capable of blending street view panorama with high quality and faultless color continuity, which obviously outperforms the open-source software Enblend and the popularly used commercial software PTGui.

2. Approach

For an arbitrary image, both the Probability Density Function (PDF) and the Cumulative Distribution Function (CDF) with respect to luminance histograms describe the statistical characteristics and color distribution features in a more visually stimulating way, which are extremely informative and worth studying. In probability theory and statistics, the PDF is a function that describes the relative likelihood for a random variable to take on a given value while the CDF describes the probability that a real-valued random variable with a given probability distribution will be found to have a value less than or equal to this random variable. In our method, the random variables in both PDF and CDF denotes the intensities of pixels in an 8-bit grayscale image. The main idea of our proposed method is that the PDF and CDF of the overlap regions in the first image should be approximately equal to those in the second image overlapped with the first image with respect to their curve trends. Our approach is mainly composed of following four parts:

  • Automatic Contrast Adjustment: Multiple channels of individual images are automatically adjusted in contrast to make sure that multiple images have the similar contrast, which will reduce the brightness differences between images.
  • Finding Extreme Points: The extreme points of luminance histograms in the overlap areas are picked out as feature points to represent statistical characteristics of the image.
  • Matching Extreme Points: we correct the color of these two images to ensure that the extreme points in the luminance histograms (or PDFs) computed from the overlap regions of the color-corrected images are the same. So we match these extreme points with two predefined conditions. The CDFs also contribute to it when the number of extreme point matches is less than a predefined value.
  • Color Correction and Multi-Band Blending: The intensities of all the pixels are corrected according to those matching points with certain rules. Then the corrected images are blended by Laplacian pyramid blending method with mask images both related to valid regions and seamlines of the corresponding input images.

The details of algorithm are described in the paper.

3. Experimental Results

3.1. Information of Experimental Data

To evaluate the effectiveness and superiority of our proposed method, we compared our blending results with those of the opensource software Enblend and the commercial software PTGui, which were both popularly used in the field of panoramic photography. The street view images were obtained by a mobile vehicle platform with 6 cameras with wide-angle lens and precisely aligned afterwards. The images obtained from Leador and Tencent are listed in Fig. 1 and Fig. 2 respectively.

Fig. 1. Original images from Leador mobile vehicle platform.


Fig. 2. Original images from Tencent mobile vehicle platform.

3.2. Comparative Experiments

Comparative experiments were conducted by using Enblend, PTGui and our proposed method respectively. Our method was implemented with C++ under Windows and tested in a computer with an Intel Core i7-4770 at 3.4GHz with 16GB RAM. The visual results are expressed below:



Fig. 3. The visual comparison between blended panoramas by Enblend (in the first and thrid rows) and our proposed method (in the second and fourth rows).



Fig. 4. The visual comparison between blended panoramas by PTGui (Top) and our proposed method (Bottom).

Citation

Yaping Liu, Jian Yao*, Mi Zhang, and Li Li. Color Correction and Image Blending for Panorama Stitching via Extreme Point Matching of Luminance Histogram, submitted to IEEE International Conference on Image Processing (ICIP), 2015.