Duration of the Thesis: 6 months
Completion: October 2012
Supervisor: Dipl.-Ing. Konrad Wenzel
Examiner: Prof. Dr.-Ing. Norbert Haala
For a long period of time airborne LiDAR was the ideal method to acquire 3D point clouds which are necessary for generating DSMs. This method is able to deliver a high vertical accuracy (e.g. below 50 mm) even from high flight altitude which enables efficient measurements with low occlusions. Although, LiDAR is advantageous for providing high vertical accuracy, the main drawbacks of the system are limited measurement density as well as the high costs associated with the sensor that is used to measure the necessary point cloud data.
Nowadays, available aerial cameras have a strong geometry and can acquire high resolution aerial images. Also by advances in image matching algorithms, automatic height data computation by stereo methods became a proper alternative for the DSM generation. One suitable option for the advanced image matching algorithms is the Semi-Global Matching (SGM) stereo method that can compute very dense point clouds. As aerial images are acquired for many areas, they can be attainable with reasonable prices. These advantages and also limitations of the LiDAR technology (i.e. measurement density, sensor price), motivates to generate DSMs by means of dense image matching.
Nevertheless, DSMs which are extracted from image matching point clouds suffer from occlusions, outliers and noise. Therefore, the aim of this thesis is firstly, extracting DSMs from dense point clouds generated by the Semi-Global Matching (SGM) stereo method, and secondly, improving the extracted DSMs in order to overcome the shortfalls of occlusions, remove outliers and reduce noise. Here, special interest is paid to the enhancement of the buildings’ edges.
In this work, the necessary methods and algorithms have been implemented in C++ with the use of OpenCV library as well as MATLAB. The sequence of the developed operations in the methodology is shown in Figure 2.1 . Throughout this thesis, several datasets from different sources have been processed and analyzed; three dense point clouds generated by means of the Semi-Global Matching (SGM) stereo method, and two point clouds acquired by LiDAR.
|
| Figure 2.1: Flowchart of the methodology pipeline |
DSMs are stored in raster format which represents the height information on regularly spaced grid elements. The first task in this step is to extract a raster DSM out of a point cloud. And then, the height values in the extracted DSM are converted into gray values in order to visualize the DSM as a gray scale image. The specification of the processed point clouds and the extracted DSMs in this thesis are summarized in Table 1.
| Table 1: Specification of the extracted DSMs from different datasets |
 |
The aim of this step is to interpolate those cells in the extracted DSM that have no height values. The interpolation method which is used here is the interpolation on paths. This approach preserves the real edges of buildings as well as it does not use the height values of outliers for interpolating the occluded areas.
 |
| a) A part of a DSM before interpolation |
 |
| b) Interpolated using the interpolation on paths |
| Figure 2.2: Preserving the real edges of buildings using the interpolation on paths method |
In order to improve the DSM, one of the most important steps is to remove outliers. Naturally, outliers have certain height differences comparing to their correct surrounding cells. These height differences are the clues to detect outliers using a function called filterSpeckles from the OpenCV library. Since outliers exist in different blob sizes with various height differences comparing to their surrounding cells, in order to detect them efficiently, the function is applied twice.
 |
| Figure 2.3: DSM before applying filterSpeckles (left) and the detected outliers (right) |
In this step, the DSM is classified and segmented into terrain and off-terrain classes. The segmentation result is used later in the robust interpolation as a constraint in order to separate the interpolation between the terrain and off-terrain surfaces. The segmentation method which is used here consists of detecting the cells describing the borders of off-terrain objects such as building and vegetation, and then filling these borders to get the segments.
For detecting the off-terrain edges, the differentiation of Gaussian Blur is used, which works automatically and does not need any operator interaction. This differentiation is thresholded using a certain height value which is constant for all the datasets. Once the edges of off-terrain objects have been detected, the segmentation step can start. For this purpose, the detected edges and the DSM are analyzed and processed simultaneously. Figure 2.4 shows the results of the edge detection and segmentation for a DSM from Kirchdorf data set.
 |
| Figure 2.4: DSM from the Kirchdorf dataset (left), the detected edges (middle) and the segments (right) |
The segmentation results that have been achieved in the previous step contain segment residuals, dents in the edges and holes inside the off-terrain segments. Since these segmentation results are used later as a constraint in the robust interpolation, the quality of the interpolated DSM directly depends on the condition of the segments. Therefore, in this step a set of morphological filters are applied to refine the segmentation results.
For refining the segmentation results, firstly morphological opening is applied in order to erase the punctual segments remaining on the terrain. Secondly, morphological closing allows filling the dents and small holes in the off-terrain segments. And finally, hole filling algorithm is used to fill the remaining gaps inside the off-terrain body.
 |
| Figure 2.5: Segments for a LiDAR DSM before (left) and after the morphological filtering (right) |
After applying the morphological filters, the segmentation results have been improved significantly. However, in order to have high quality DSMs, the segments need to be enhanced more. As the edges of the off-terrain segments are still jagged and noisy, in this step the Douglas-Peucker algorithm is applied on the segments to smooth the edges.
This algorithm simplifies a polygon by reducing the points in polylines forming the polygon based on a specified smoothing threshold. In the case of DSM, the Douglas-Peucker algorithm is applied on the off-terrain segments. Borders of the off-terrain segments are considered as polygons, thus this algorithm can smooth these borders.
 |
| Figure 2.6: After the morphological filtering (left), approximated polygons (middle) and updated segments (right) |
In this step, the extracted DSM is interpolated based on the enhanced segments so that it is possible to overcome the shortfalls of occlusions. In the Inverse Distance Weighting (IDW) interpolation method which is used here, the segments serve as a constraint which helps to interpolate the terrain and off-terrain cells separately. Since, the off-terrain segments have been improved by means of the morphology filtering and applying the Douglas-Peucker algorithm, this interpolation approach delivers a refined DSM with smoothed off-terrain edges as shown in Figure 2.7.
 |
| Figure 2.7: DSM before interpolation (left), the improved segments (middle) and DSM after the robust interpolation (right) |
In this section, different results after applying all the improving steps are illustrated. Firstly, a part of a DSM that has been refined using the explained approach in this text is shown versus the same part of the DSM before the improvement (see Figure 3.1). Secondly, produced orthophotos from DSMs before and after the refinement are represented in Figure 3.2. Lastly, in Figure 3.3, 3D visualization of an improved DSM is presented where the edges of buildings have been enhanced comparing to the same edges before the improvement.
 |
| Figure 3.1: DSM interpolated using a traditional method (left) and DSM after the robust interpolation (right) |
 |
| Figure 3.2: Produced orthophoto using a DSM before (left) and after the improvement (right) |
 |
| Figure 3.3: 3D visualization of a DSM before (left) and after the improvement (right) |
Compare to the Gauss filter, the Fan filter can reduce more stripes.
The approach which has been presented in this thesis can extract DSMs from dense point clouds, and deliver refined DSMs. By means of this method, the edges of the off-terrain objects are preserved and improved and terrain and off-terrain invalid cells are interpolated separately. However, applying the Douglas-Peucker algorithm sets the focus on preserving and smoothing the edges of buildings and not vegetation. As a result of using this algorithm, correct parts of vegetation with round shapes are removed and destroyed. Therefore, in the case of applying the Douglas-Peucker, the generated DSMs can be used mainly in applications such as 3D building reconstruction where the smoothed edges of buildings are demanding and vegetated areas are not in interest.
The main objective of this work was to derive and improve high resolution DSMs from dense point clouds generated by the Semi-Global Matching (SGM) stereo method. However, point clouds acquired by LiDAR have been processed and analyzed through this thesis. Naturally, point clouds measured by LiDAR do not suffer from outliers and occlusion as much as image matching data. Nevertheless, as the enhanced segmentation results are used in the interpolation process as a constraint, such LiDAR data still can be improved.
Despite all the improvements, quality of the generated DSMs is directly dependent on the quality of the input point clouds. Through the image matching process, larger number of images with strong geometry increases the redundancy, and consequently the generated point clouds have less occlusion, outliers and noise. Using point clouds with less outliers to extract DSMs, definitely delivers refined off-terrain segments after all the improving steps. Since this approach uses the enhanced segments as a constraint in the robust interpolation step, having good quality segments guarantee to have improved DSMs.