Segmentation-Based Classification for 3D Point Clouds in the Road Environment

1. Abstract

The 3D point cloud classification in urban scenes has been widely applied in the fields of automatic driving, map updating, change detection, etc. Accurate and effective classification of mobile laser scanning (MLS) point clouds remains a big challenge for these applications. In this paper, we propose a unified framework to classify 3D urban point clouds acquired in the road environment. At first, an efficient 3D point cloud segmentation approach is applied to generate segments for further classification. This is achieved by using the Pairwise Linkage (P-Linkage) algorithm for the initial point clouds segmentation followed by our proposed two-step post-processing approach to improve the original segmentation results for accurate classification. Secondly, a set of novel features are extracted from each segment and an effective classifier for training and testing is used. The good performance of the extracted features is determined by employing three popularly used classifiers, Support Vector Machine (SVM), Random Forests (RF) and Extreme Learning Machine (ELM), respectively. Thirdly, the contextual constraints among objects are used to further refine the classification results based on segments via graph cuts. A set of experiments on our own manually labelled dataset show that our proposed framework can effectively segment the testing point clouds. On the test dataset, the initial classification can reach a high precision of 80.8%-92.9% and a good recall rate of 77.5%-79.2%, respectively. After the classification refinement via graph cuts, the precision and recall rate are increased about 0.3% and 3.1%, respectively. These experimental results convincingly prove that our proposed framework is effective for classifying 3D urban point clouds acquired by a vehicle LiDAR system in the road environment.

2. Our Approach

The overall work-flow can be separated into three stages, as illustrated in Fig.1.

Fig. 1. The overview flowchart of our proposed unified framework for classifying 3D urban points clouds acquired in the road environment.

The first stage is to segment the original unstructured 3D point cloud by using the P-Linkage algorithm, which is a recently proposed region-growing-based hierarchical segmentation algorithm. After that, in order to solve the problems caused by over-segmentation, such as the reduction of the quality of the segment features and the increment of noise, a two-step post-processing is proposed: 1) the segments of cars, trees and curbs are grouped by the connected component analysis; 2) nearly co-linear segments are merged. The two-step post-processing procedure can be depicted as Fig.2.

Fig. 2. An illustration of our proposed segmentation post-processing strategies: (a) the original P-Linkage segmentation result; (b) the ratios of scatter points for each segment, which range from 0 to 1; (c) the candidate segments in the first-step post-processing; (d) the ratios of linear points for each segment, which range from 0 to 1; (e) the candidate segments in the second-step post-processing; (f) the final segmentation result after two-step post-processing.

In the second stage, we extract a set of features from each segment for training and testing by using an effective classifier. For comparison, we employ three classifiers (SVM, RF and ELM) to classify the point clouds, respectively. The features defined in this research are listed in Table 1.

Table 1 A list of features extracted from a point cloud segment.

In the third stage, due to that the classifier cannot give smooth and high accurate results, we use the contextual constraints among objects via the graph cuts energy minimization algorithm to further refine the initial classification. The procedure can be depicted as Fig.3.

Fig. 3. An illustration of classification refinement via graph cuts: (a) the initial classification results; (b) the region growing result based on the same initial classification labels where different colors represent different objects; (c) the separation of reliable and unreliable objects according to their point sizes where unreliable objects are represented by black points; (d) the classification refinement of unreliable points for each reliable object by applying the graph cuts optimization; (e) the finally refined classification results after optimization.

3. Results

The point cloud dataset used in this paper was captured form Huangshi city of Hubei Province in China, and this data was acquired by using the SICK LMS511 laser scanners mounted on a vehicle. We constructed the ground truth dataset by manually labeling each 3D point with corresponding object class. By observing the original data, we choose the following nine class labels: ground, buildings, cars, trees, curbs, fences, street lights, telegraph poles and electric wires. The dataset finally consists of 7 separately continuous point clouds. The informations of the dataset are presented in Table 2.

Table 2 The informations of manually labeled class ground truth point cloud dataset.

3.1. Segmentation

To test the robustness of the proposed point cloud segmentation method, we apply it on 3 sets of laser scanner point clouds (S₁, S₂ and S₃ as shown in Table 2) from our built dataset, which consist of 1,050,774, 1,074,792 and 975,256 points, respectively. The informations and segmentation results in detail are summarized in Table 3. In order to show the segmentation results clearly, Fig.4 presents the segmentation results of two regions before and after post-processing. These two regions are selected from the point cloud set S₂, which are named as region I and region II, respectively.

Table 3 Segmentation statistical results of 3 sets of point clouds.

(a) the original P-Linakge segmentation result for region I from the point cloud set S2	(b) the segmentation result after post-processing for region I from the point cloud set S2
(c) the original P-Linakge segmentation result for region II from the point cloud set S2	(d) the segmentation result after post-processing for region II from the point cloud set S2

Fig. 4. The segmentation results before and after post-processing for the region I and II from the point cloud set S₂.

3.2. Initial Classification

We conducted two comparative experiments to verify the effectiveness of our proposed approach. To validate the effectiveness of our proposed post-processing strategies for classification, in the first experiment, we compared the classification results based on the segments before and after post-processing on the three testing point cloud sets using the SVM classifier, as shown in Table 4.

Table 4 The comparison results between the classification of original P-Linkage segmentation and the classification based on the segmentation after post-processing for 3 testing point clouds.

In addition, in order to prove that our extracted features are also applicable to other classifiers except for SVM, we also conducted a comparative experiment among three different kinds of classifiers: SVM, RF and ELM. The classification performances for three different classifiers on the three testing sets are showed in Table 5.

Table 5 The classification performances for the three testing point cloud sets using three kinds of classifiers (SVM, RF and ELM).

3.3. Optimization

Fig.5 shows some close-ups of the final classification results after applying the graph cuts optimization algorithm. The generated segmentation result shows clearly distinguished labeling results for different objects. To quantitatively evaluate our proposed optimization approach, we compared the classification performances before and after optimization. The precisions and recall rates of three testing sets for the initial classification and the classification after optimization are reported in Table 6.

(a)	(b)
(c)	(d)
(e)	(f)

Fig. 5. Close-ups of the classification result after optimization: (a) the street view with grounds, buildings, fences, trees, street lights, cars and telegraph poles; (b) the street view with grounds, fences, buildings, trees, telegraph poles, street lights and electric wires; (c) and (e) the street view with all the 9 object classes; (d) and (f) the local details of partial regions in (c) and (e), respectively.

Table 6 The comparative results between the initial classification and the classification after optimization for three testing sets.