In this paper, we present a robust method to efficiently create a globally consistent and seamless mosaic from aerial images. Firstly, a globally consistent registration strategy is proposed to align the aerial images in a common coordinate system, which combines the affine model with the homographic model effectively. To suppress the error accumulation of perspective deformations via a sequential set of aerial images taken from a wide-range region, we proposed to initially align each image by an affine model and then perform a homographic refinement in groups to increase the global consistency. Secondly, to efficiently conceal the parallax between aligned images in regions with large depth differences where it is impossible for recovering a highly accurate consistent image registration, a novel optimized seamline detection algorithm in the graph cuts energy minimization framework is proposed to find optimal seamlines within overlapped regions for image mosaicking through rounding visually obvious foreground objects. Finally, experimental results on several representive image sets illustrates the superiority of our proposed approaches.
Our approach is mainly composed of two parts: global alignment and parallax removal. The global alignment module aims at mosaicking sequential images with keeping both accurate alignment and global consistency, and the parallax removal module is employed to conceal the parallax between aligned images in regions with large depth differences.
Global alignment module:
Parallax removal module:
The details of algorithm are described in the paper.
To sufficiently test the performance of our proposed algorithm for global alignment of aerial images, we chose a set of 90 aerial images in three strips with the down-sampling image size of 746×480 and a set of 48 UAV images in three strips with the down-sampling image size of 800×533 for evaluation. These two test image sequences are mainly different in flight altitudes and stability of image acquisition platforms, in which there exist an overlapping rate of about 60% in adjacent images.
Part of the aerial sequence images and the UAV sequence images are listed in Fig. 1 and Fig. 2 respectively.
In the period of obtaining initial alignment, the selection of the transformation model among rigid, affine and homographic models can make some differences to the final mosaicking result, which are compared through doing experiments on the two datasets above. The visual results and numerical results are expressed below:
Visual results(Table. 1. Click the Thumbnails to view the original images ):
Numerical results(Fig. 3, Fig. 4 and Table 2):
|Initial Models||Aerial Images||UAV Images|
We compared our approach with a commercial software named PTGui. The visually comparative results of mosaicking aerial images and UAV ones are illustrated in Fig. 5 and Fig. 6 respectively,in which the optimal seamline detection and image blending are not applied. You can view the original images by click the images.
Notice that the cyan box is labeled for showing the position of the reference image. Besides, the results of PTGui used for comparisons are preview versions in the processing procedure of PTGui( as you can see the ruler lines on the images), because the mosaicking result from overlapping each image directly is not available.
Based on our proposed optimal seamline detection via graph cuts, the extracted seamlines in overlapped regions between images can visually conceal the parallax.
Fig. 7 shows the seamline detection result of our proposed algorithm on 6 images of a single strip,
where our optimal seamlines successfully bypassed the tall buildings where there exist large parallaxes between images. You can view the original images by click the images.
To further validate the superiority of our approach in global alignment for image mosaicking, we tested another set of aerial images which is selected from the same data set as the 90 aerial images mentioned above. This set of aerial images, 90 images of 3 strips, were mainly captured on the building-compact districts. Besides, we also try to mosaic images selected from a single strip.
The mosaicking results of a 3-strips and two single-strip are showed in Fig. 8, Fig. 9 and Fig. 10, and you can view
the original images by click the images. What’s more, the numerical results are available in Table 3.
|Fig. 8 Mosaicking result of the 3-strips|
|Fig. 9 Mosaicking result of the Single-Strip1|
|Fig. 10 Mosaicking result of the Single-Strip2|
Besides, we did experiments on two sets of images from two single strips with different values of λ in the cost function, and the two single strips are Single-Strip1 and Single-Strip2 mentioned above, which represent the scene with large depth differences and the scene of small depth differences, respectively. The mean RMS of reproject-error of mosaicking result with different λ are showed in Fig. 11, and several representative mosaic images with different value of λ are compared in Table 4 to show the difference in global consistency(detail differences are labelled by red boxes).
Judged by both the global consistency of mosaicking result and alignment precision, we find that the value between 0.03 and 0.11 can always give a good mosaicking result for both case, and the optimal one should be set according to the property of data, such as 0.06 and 0.09 are respectively chosen for Single-Strip1 and Single-Strip2 in our experiments.