Structural assessment using terrestrial laser scanning point clouds

Abstract

Purpose

Terrestrial laser scanning (TLS) point clouds have been widely used in deformation measurement for structures. However, reliability and accuracy of resulting deformation estimation strongly depends on quality of each step of a workflow, which are not fully addressed. This study aims to give insight error of these steps, and results of the study would be guidelines for a practical community to either develop a new workflow or refine an existing one of deformation estimation based on TLS point clouds. Thus, the main contributions of the paper are investigating point cloud registration error affecting resulting deformation estimation, identifying an appropriate segmentation method used to extract data points of a deformed surface, investigating a methodology to determine an un-deformed or a reference surface for estimating deformation, and proposing a methodology to minimize the impact of outlier, noisy data and/or mixed pixels on deformation estimation.

Design/methodology/approach

In practice, the quality of data point clouds and of surface extraction strongly impacts on resulting deformation estimation based on laser scanning point clouds, which can cause an incorrect decision on the state of the structure if uncertainty is available. In an effort to have more comprehensive insight into those impacts, this study addresses four issues: data errors due to data registration from multiple scanning stations (Issue 1), methods used to extract point clouds of structure surfaces (Issue 2), selection of the reference surface S_ref to measure deformation (Issue 3), and available outlier and/or mixed pixels (Issue 4). This investigation demonstrates through estimating deformation of the bridge abutment, building and an oil storage tank.

Findings

The study shows that both random sample consensus (RANSAC) and region growing–based methods [a cell-based/voxel-based region growing (CRG/VRG)] can be extracted data points of surfaces, but RANSAC is only applicable for a primary primitive surface (e.g. a plane in this study) subjected to a small deformation (case study 2 and 3) and cannot eliminate mixed pixels. On another hand, CRG and VRG impose a suitable method applied for deformed, free-form surfaces. In addition, in practice, a reference surface of a structure is mostly not available. The use of a fitting plane based on a point cloud of a current surface would cause unrealistic and inaccurate deformation because outlier data points and data points of damaged areas affect an accuracy of the fitting plane. This study would recommend the use of a reference surface determined based on a design concept/specification. A smoothing method with a spatial interval can be effectively minimize, negative impact of outlier, noisy data and/or mixed pixels on deformation estimation.

Research limitations/implications

Due to difficulty in logistics, an independent measurement cannot be established to assess the deformation accuracy based on TLS data point cloud in the case studies of this research. However, common laser scanners using the time-of-flight or phase-shift principle provide point clouds with accuracy in the order of 1–6 mm, while the point clouds of triangulation scanners have sub-millimetre accuracy.

Practical implications

This study aims to give insight error of these steps, and the results of the study would be guidelines for a practical community to either develop a new workflow or refine an existing one of deformation estimation based on TLS point clouds.

Social implications

The results of this study would provide guidelines for a practical community to either develop a new workflow or refine an existing one of deformation estimation based on TLS point clouds. A low-cost method can be applied for deformation analysis of the structure.

Originality/value

Although a large amount of the studies used laser scanning to measure structure deformation in the last two decades, the methods mainly applied were to measure change between two states (or epochs) of the structure surface and focused on quantifying deformation-based TLS point clouds. Those studies proved that a laser scanner could be an alternative unit to acquire spatial information for deformation monitoring. However, there are still challenges in establishing an appropriate procedure to collect a high quality of point clouds and develop methods to interpret the point clouds to obtain reliable and accurate deformation, when uncertainty, including data quality and reference information, is available. Therefore, this study demonstrates the impact of data quality in a term of point cloud registration error, selected methods for extracting point clouds of surfaces, identifying reference information, and available outlier, noisy data and/or mixed pixels on deformation estimation.

Citation

Truong-Hong, L., Lindenbergh, R. and Nguyen, T.A. (2022), "Structural assessment using terrestrial laser scanning point clouds", International Journal of Building Pathology and Adaptation, Vol. 40 No. 3, pp. 345-379. https://doi.org/10.1108/IJBPA-04-2021-0051

1. Introduction

Laser scanning, also known as light detection and ranging (LiDAR), has been used to quickly and accurately acquire three-dimensional (3D) topographic data of visible surfaces. The fundamental principle of LiDAR involves a laser beam to measure the distance from the instrument to a surface of an object based on the time of travel between signal transmission and reception called a laser pulse, and a 3D coordinate of an intersection point between the laser pulse and the surface is computed. The output datasets consisting of x-, y-, z-coordinates associated with other attributes are commonly referred to as a 3D point cloud. One of the LiDAR units is a terrestrial laser scanner (TLS), the laser sensor operating from the ground capturing great details of surfaces of objects with millimetre accuracy. Thus, this type of laser scanning data has been used in civil engineering like construction progress monitoring, and dimensional quality control (Son and Kim, 2010), construction management (Bosché, 2010), road modelling (Boyko and Funkhouser, 2011; Truong-Hong et al., 2019) and disaster planning (Laefer and Pradhan, 2006). Interestingly, it is also being applied for structural assessment, for example, creating geometric models for Finite element (FE) mesh generation (Truong-Hong et al., 2012; Kassotakis et al., 2020), determining surface damage (Mizoguchi et al., 2013; Sedek and Serwa, 2016) and deformation measurement (Bosché and Guenet, 2014; Truong-Hong and Lindenbergh, 2019). The deformation measurement has attracted both academia and professional communities because laser scanning can give high accuracy of deformation of an entire surface of a structure instead of discrete locations on the surface obtained from traditional survey methods.

In structural assessment–based deformation analysis, both relative and absolution deformation estimation are required to assess behaviour of individual members and an entire structure. Particularly, for large-scale structures, including dams, tunnels and bridges, the absolution deformation may give information on the interaction between the structure and the surrounding environment. For example, the absolution deformation of a dam may be related to the water level of a reservoir (Yigit et al., 2016), or a tunnel movement may result in ground settlements–induced building damage (Giardina et al., 2019). To estimate relative deformation, geometric data of the structure in the same local coordinate system can be used (Lindenbergh and Pietrzyk, 2015; Jiang et al., 2021), while a geodetic network is required to record deformation/movement of the structure when estimating absolute deformation (Kavvadas, 2005). When laser scanning is used to estimate the absolute deformation, a point cloud of a single scan station is registered into the geodetic coordinate system, which may result in increasing the total error budget (Lichti Derek et al., 2005). Moreover, for large-scale structures, laser scanning often captures the structure at low temporal frequency (e.g. monthly or yearly) because of budget limitations. Alternative methods, for example global navigation satellite system (GNSS) (Montillet et al., 2016; Xi et al., 2021), are often installed on a structure to record movement at specific locations with high frequency at lower costs. However, these methods measure at only fixed locations of the structure and local behaviour of the structure can be missed (Harmening and Neuner, 2020). This paper presents the use of a laser scanner to estimate deformation of small-scale structures, and particularly focuses on relative deformation, such that georeferencing of the point cloud is excluded.

A typical workflow for deformation analysis consists of four main steps: data acquisition, data registration, point cloud extraction and deformation estimation. Several studies have proposed and discussed solutions for each individual step. For example, first, in identifying the influence of scanning parameters on data quality, Soudarissanane et al. (2011) and Laefer et al. (2009) remarked that the precision of a point cloud decreases when the incidence angle and scanning measurement increases, particularly for incidence angles larger than 70 degrees. Second, Cheng et al. (2018) gave a brief overview of data registration methods and reported that the point cloud registration error depends on the quality of data points, and can vary between 3 mm and 22 m. Third, in discussing segmentation methods, Xie et al. (2020) addressed that the accuracy of the segmentation relies on various parameters, for example, region growing depends on the growing criteria and selected seeding points. Finally, in a survey of the use of TLS point clouds for structural deformation measurement, Truong-Hong and Laefer (2014) noted that errors of deformation estimation–based point clouds varied from 9 mm to 30 mm. These studies discussed advantages and disadvantages of methods for specific applications, for example, monitoring structural deformation or measuring vertical clearance of a bridge. Considering various effects of the methods used in a complete workflow for deformation measurements based on TLS point clouds has not been addressed, the main contributions of the paper are as follows:

1. investigating point cloud registration error affecting resulting deformation estimation;

2. identifying an appropriate segmentation method used to extract data points of a deformed surface;

3. investigating a methodology to determine an un-deformed or a reference surface for estimating deformation;

4. proposing a methodology to minimize the impact of outlier, noisy data and/or mixed pixels on deformation estimation.

Finally, a bridge abutment, building and steel frame of a storage oil tank were selected as case studies to demonstrate this study.

2. Related work

For the last two decades, laser scanning has been widely used to acquire topographic and texturing information of surfaces of structures for inspection and assessment. To date, most studies use laser scanning point clouds to identify surface deficiencies (e.g. water leakage or moisture), determine material loss (area and volume losses) and deformation measurement, which are investigated in this section, while other applications of the point cloud can refer to Laefer (2020).

For detecting surface damage appearing through texturing information, for example, water bleeding, chemical attack and corrosion/crusting, intensity and/or colour of a point cloud were used. For example, Armesto-González et al. (2010) used machine learning–based intensity images of 3D point cloud to detect moisture of surfaces, in which the point clouds acquired from various scanners (e.g. FARO Photon, TRIMBLE GX200, and RIEGL-Z390i) were used to investigate influence of intensity variance on resulting detection. Similarly, Suchocki et al. (2020) proposed segmentation-based intensity values of a point cloud to extract an area subjected to moisture and biofilm. Interestingly, the authors reported that damage detection based on the point cloud from the phase shift scanner is more sensitive than that from the time-of-flight scanner. Additionally, Tan et al. (2016) found a linear model between the point cloud and water content intensity when detecting water leakage of a tunnel.

For example, to estimate volume loss of a reinforced concrete beam column, Olsen et al. (2010) proposed the crossing section method to determine a polygon fitting a perimeter of a cross-section, and the volume loss was computed from the polygon areas and interval distances between adjacent cross-sections. Teza et al. (2009) detected damage area by analysing Gaussian curvature of sub-area of a region of interest, but missed detection can occur if the noise of the point cloud exceeded 8 mm. Similarly, distance and gradient computed from rectangular grids of a selected area to a reference plane were used to detect defective areas of the pile cap, in which a predefined distance and gradient thresholds were used to classify damaged and undamaged grids (Liu et al., 2011). Suchocki and Błaszczak-Bąk (2019) defined that data points of crack face and the cavity of brick walls have distances to a reference surface more than 5 mm, in which the reference surface was determined from entire points of the wall. Additionally, to measure the surface loss of a steel beam due to corrosion, Truong-Hong and Laefer (2015b) used an angle criterion based on a 2D grid to extract point clouds on a boundary of a hole, and an area of the hole was then computed from a polygon fitting through the boundary points.

In addition, in attempting to use laser scanning to measure crack on a surface of a structure, Truong-Hong et al. (2016) mapped Red-Green-Blue (RGB) images on a point cloud of a masonry abutment to assist users in identifying locations of cracks. Next, Cabaleiro et al. (2017) used an alpha shape method to extract data points on a boundary of crack of a timber beam and reported that the method can extract the crack width more than 3 mm for a sampling step of 1 mm. For documenting the cracks on substructures of a bridge, Valença et al. (2017) map cracks detected from the 2D image onto a point cloud, in which the crack face is known as discontinuity of the surface is determined from the point cloud. However, as noise and spatial information of data points, crack extraction based on the laser scanning point cloud can obtain the best performance if the crack width is more significant than 5 mm.

This study aims to investigate factors that affect resulting deformation estimation. The rest of this section is restricted to investigating recent works relating to deformation measurement based on TLS point clouds. Many methods have been proposed to analyse a point cloud for change detection or deformation measurement. For example, a cloud-to-cloud method measured deformation through a distance from any point of the reference surface to its nearest neighbour point in the sampling surface (Girardeau-Montaut et al., 2005). Additionally, Lague et al. (2013) proposed a multiscale model to model cloud comparison for change detection through an average distance between two sub-clouds from the reference and sampling surfaces, in which the cylinder neighbourhood is used to extract the sub-cloud. Those methods are mainly used for topographic change detection. Following this, the work of using point clouds for measuring deformation of the civil engineering structure is surveyed.

Bosché and Guenet (2014) inspected the flatness of the building floor by mapping a 3D point cloud to an as-design building information modelling (BIM) model to measure the deviation between the point cloud and BIM by using a straightedge and F-numbers methods. Tang et al. (2011) proposed three algorithms consisting of range filtering, deviation filtering and sliding window to inspect floor defects through deviations between the points and the reference surface. The main difference between these methods is the different methodologies used to denoise the point clouds of the surfaces. Li et al. (2020) determined the flatness of a concrete surface by distances between the points of the surface to the reference surface estimated from 80% points of the surface, in which random sample consensus (RANSAC) was carried out (Schnabel et al., 2007).

In measuring deformations of a heritage structure, Bertacchini et al. (2010) used a centroid of each horizontal slice along a vertical direction of the tower to determine deviation from verticality (out-of-plumb) of the Asinelli Tower in Bologna, in which the interval thickness of the slice was 10 cm. Pesci et al. (2013) investigated the effects of noisy data and selected reference information in evaluating building deformation due to seismic activity by using TLS point clouds. The buildings were scanned from various scanning ranges and angles, while point-to-primitives (e.g. planes or cylinders) were computed to determine building deformation. The study recommended that the high incidence angle must be quantified before using point clouds for measuring deformation, and the primitive shape strongly depends on the spatial distribution of the point cloud. Moreover, based on knowing the shape of a tower of the Ayutthaya temples, Thailand, the cylinder fitting was used to reconstruct the tower from its point cloud, and the axis along a longitudinal direction of the cylinder is used to measure a tilt angle of the tower (Bhadrakom and Chaiyasarn, 2016).

To measure the deformation of an individual girder of a bridge, Lichti et al. (2002) compared fitting lines of the top and bottom wood stringers of the bridge under unloaded and loaded conditions to determine vertical displacements. Moreover, Paffenholz et al. (2008) subdivided a point cloud of the girder surface into 2D cell-grids with cell size of 0.25 m and used the median of z coordinates of each cell to determine vertical displacements of the bridge in a static loading test. Zogg and Ingensand (2008) monitored deformations of the Felsenau Bridge subjected to a static load of 54 tons performed at several viaduct sections. Third-party software was used to determine vertical displacements by comparing the 3D point cloud from the unloaded condition against the loaded condition. The TLS-based results were no more than 3.5 mm larger than ones based on precision levelling. Moreover, in measuring a vertical clearance of bridges, Riveiro et al. (2013) fitted a point cloud of a beam camber and pavement by a polynomial curve and a plane, respectively, while the vertical clearance was the difference of z values computed from the curve and the plane. Truong-Hong and Lindenbergh (2019) proposed point–surface, point–cell and cell–cell methods, in which the high order polynomial surfaces or multiple planar patches were used to describe the bottom surfaces of the girders, while the vertical clearance is defined as distances from the points describing a road surface to the bottom surfaces of the girders.

On another application, Van Gosliga et al. (2006) measured the deformation of a bored tunnel by comparing the point cloud to a fitting cylinder model of the tunnel derived from a sub-data set. To measure displacements of a load-test beam, similar to a cell-to-cell method proposed by Truong-Hong and Lindenbergh (2019), Cabaleiro et al. (2020) divided a bottom surface into rectangular meshes, and deformations are distances between the centroid of the element mesh from different load stages, in which the centroid of the element is computed, and the data points within the mesh. The authors addressed that an error of the deformation measurement is less than ±0.02 mm when a high precision scanner with the accuracy of ±0.025 mm is used in acquiring the point cloud.

In summary, although a large amount of the studies used laser scanning to measure structure deformation in the last two decades (Mukupa et al., 2017), methods mainly applied to measure change between two states (or epochs) of the structure surface and focus on quantifying deformations-based TLS point clouds. Those studies proved that a laser scanner could be an alternative unit to acquire spatial information for deformation monitoring. However, there are still challenges in establishing an appropriate procedure to collect a high quality of point clouds and developing methods to interpret the point clouds to obtain reliable and accurate deformation, when uncertainty, including data quality and reference information, is available. Therefore, this study demonstrates the impact of data quality in a term of point cloud registration error, selected methods for extracting point clouds of surfaces, identifying reference information, and available outlier, noisy data and/or mixed pixels on deformation estimation.

3. Structural assessment-based deformation

After data acquisition and preprocessing, a deformation estimation–based point clouds procedure consists of two main steps: (1) extracting a point cloud of a surface of a structural component, (2) estimating the deformation. The first step is executed through point cloud segmentation methods, for example, region growing (Rabbani et al., 2006), Hough transforms (Hough, 1959), and RANSAC (Schnabel et al., 2007). However, due to complexity, difficulty in implementation and selecting parameters, the region growing and RANSAC are primarily selected to implement and use in practice (Cloudcompare, 2021; Leica Geosystems, 2020). As such, region growing-based methods (Vo et al., 2015; Truong-Hong and Lindenbergh, 2020) and RANSAC (Schnabel et al., 2007) implemented (Cloudcompare, 2021) are used to extract data points of surfaces of a structure to investigate an impact of segmentation methods on accuracy of deformation estimation.

3.1 Region-growing based segmentation method

Region growing–based methods are an iterative process to extract points having deviations of salient features (e.g. orthogonal distances and normal vectors) satisfying predefined criteria, and these points are recognized as the same segment. A popular region growing segmentation is the work of Rabbani et al. (2006), who used two salient features (a normal vector and residual) of data points as criteria for a growing process. In this implementation, for each point, the normal vector is normal of a plane fitted through neighbouring points, and the residual is the root mean square distances from the neighbour points to the fitting plane, in which the neighbouring points can be extracted either k-nearest neighbour (kNN) or radius neighbour search. The growing process starts with the point having the smallest residual as an initial seeding point, and its neighbour points are known as the same region with the seeding point if deviations of their normal vectors satisfy a predefined angle threshold. Any adding points having a residual less than a predefined residual threshold is considered as the seeding point in the next iteration. The process iteratively extracts the points for the region until no points are added to the region, and is completed when all points are examined.

However, region growing segmentation is required to determine salient features for each point, and the region growing process must check every point, which imposes intensive computation (Vo et al., 2015). As such, in this study, to avoid intensive computation, cell-based and voxel-based region growing methods(CRG and VRG) (Vo et al., 2015; Truong-Hong and Lindenbergh, 2020), which are basically similar to the region growing segmentation proposed by Rabbani et al. (2006), are implemented, in which the region growing process is based on local planes within cells for CRG or voxels for VRG. A point cloud was first decomposed into small cells/voxels in these methods, and points within each cell/voxel were subsequently used to extract local planes. Next, the local planes associated with salient features (e.g. a normal vector and residual) were used as input data in CRG and VRG to recognize points of surfaces. As the local planes within the cells or voxels were used, an additional condition known as the deviation of distances between local planes was included in determining if the patches or voxels were to be added to the same region of a seeding cell/voxel. Differing from point-based region growing developed by Rabbani et al. (2006), both CRG and VRG are required additional steps known as boundary refinement. That is because either cells or voxels on a boundary of a region may contain data points of adjacent regions. The step is done based on distances from the points within the boundary cells or voxels to adjoined regions. For more details of CRG and VRG, one can refer to (Vo et al., 2015; Truong-Hong and Lindenbergh, 2020). Additionally, required input parameters of CRG and VRG are cell/voxel size (c_s0 or v_s0), angle (α₀) and distance (d₀) and residual (res₀) thresholds.

3.2 Random sample consensus

A RANSAC method iteratively extracts a random sub-data set to construct a surface of a shape primitive. A resulting surface is evaluated by a score function established from the best-fitted points (or inlier points) of the shape candidate. The inlier points are determined as the points having Euclidean distances to the shape primitive less than a predefined buffer or distance threshold. Following the number of trails, the best shape primitive is the model having the highest score. Additionally, the remaining data set is subsequently used to extract the data points of other shape primitives. In this paper, the RANSAC method developed by Schnabel et al. (2007) and implemented in CloudCompare opensource (Cloudcompare, 2021), which is widely accessed and used by both professional and academic communities, is selected to extract the point cloud of surfaces of the structure. The method was developed based on the RANSAC paradigm with several refinement steps as follows:

Input data are a set of data point cloud P = {p_i = (x_i, y_i, z_i) ∈ R³, i = [1, N]} and a normal vector n_i = {nx_i, ny_i, nz_i} of each point. The algorithm randomly selects a minimum sub-data set to establish a shape primitive. For example, it requires three points to detect a plane. In addition, a function score, σ_P(ψ) = |P_ψ|, based on the number of points P_ψ is used to evaluate the candidate primitive shape (ψ). The points P_ψ are determined in two consecutive steps: (1) determining the compatible points (P′_ψ) of the candidate shape primitive ψ and (2) using a connected component analysis to eliminate spurious surfaces due to parallel surfaces of the shape available in a data set. In the former step, the compatible points (P′_ψ) are determined through two parameters including ε, the maximum distance of the points to the primitive shape ψ, and α, the deviation angle between the normal vector of the point (p_i) and the normal vector of the shape (Equation 1). Notably, this implementation requires searching the number of the compatible points P′_ψ larger than a predefined minimum number (min_ptc) to determine the candidate shape primitive (Schnabel et al., 2007).

(1)Pψ′=(p|pεP) if d(p, ψ)≤εand∠n(p)n(ψ, p)≤α

where d(p, ψ) denotes the Euclidean distance from the point p ∈ P to the primitive shape ψ, n(p) is the normal vector of the point p, and n(ψ, p) is the normal vector of the shape at the location that the point p is projected onto the shape ψ.

In the latter step, the data points P′_ψ are then projected onto the primitive shape ψ to create a bitmap with a predefined sampling resolution or pixel size (β). Next, a connected component is applied to the bitmap image to determine the largest component, and the data points corresponding to this component are known as the point P_ψ of the shape primitive.

Finally, when the best candidate shape primitive is identified, a least square method is employed to fit a geometric shape of the shape, and the compatible points having distances to the fitted shape less than 3ε are considered as the final points of the shape primitive. For more details of RANSAC implemented into Cloudcompare (2021), one can refer to the work of Schnabel et al. (2007). In summary, to detect the shape primitive from a point cloud, the RANSAC method (Schnabel et al., 2007) requires defining four parameters: (1) the minimum number of points (no_ptc_min) required to identify the candidate shape primitive, (2) the maximum distances (ε_max) between the points and the primitive shape, (3) the maximum deviation angle (α_max) between normal vectors of the points and the shape, and (4) the sampling resolution (β).

3.3 Deformation measurement

Once a point cloud of a surface of structure is extracted based on segmentation methods, for example, region growing-based methods and RANSAC mentioned above; the surface deformation is then calculated. The process consists of (1) identifying a reference surface (S_ref) and (2) computing deformation as a distance between the reference surface (S_ref) and a current surface (S_sample). The reference surface can be the structure surface when the structure starts to operate (at time t₀) or after the operation (at time t_i). For the former case, documentation of the surface is mostly not available. In this case, either the surface can be obtained either from as-design documentation or based on contextual knowledge of the structure. For example, the column is perfectly vertical, or two endpoints of the concrete beam are fixed. For the latter case, the structure surface is known or can be acquired by a terrestrial laser scanner, which is similar to the sampling surface S_sample (the structure surface at time t_j). In this study, point-to-surface is used to measure distances from the points on the sampling surface S_sample to the reference surface S_ref. The distances are herein either directional or orthogonal distances, which depends on the purpose of the applications. For example, when measuring a vertical clearance of the bridge, Truong-Hong and Lindenbergh (2019) used the directional distance in the vertical direction. Finally, the deformation herein is a relative deformation of the structure between two states (at time t₀ or t_i vs t_j). Details of the deformation measurement and impact of data and selecting information on resulted measurement are presented through three case studies in the following section. Notably, due to difficulty in a logistic, an independent measurement cannot be established to assess the deformation accuracy based on TLS data point cloud. Notably, due to difficulty in logistics, an independent measurement cannot be established to assess the deformation accuracy based on TLS data point cloud in the case studies of this research. However, common laser scanners using the time-of-flight or phase shift principle provide point clouds with an accuracy in the order of 1–6 mm (Boardman and Bryan, 2018), while the point clouds of triangulation scanners have sub-millimetre accuracy (Haddad, 2011). In practice, the triangulation scanners are rarely used because of their limitations in scanning speed and range measurement.

4. Test bed structures

In practice, the quality of data point clouds and surface extraction strongly impact resulted deformation estimation derived from laser scanning point clouds, which can cause an adverse decision when using such results for structural assessment. To have more comprehensive insight into those impacts, this study addresses four issues: data errors due to data registration from multiple scanning stations (Issue 1), methods used to extract point clouds of structure surfaces (Issue 2), selection of the reference surface S_ref to measure deformation (Issue 3), and available outlier, noisy data and/or mixed pixels (Issue 4). In Issue 1, the impact of the data quality due to registration is demonstrated by comparing deformations derived from a single scanning and multiple scanning stations. Moreover, in Issue 2, two popular point cloud segmentation methods, region growing (Vo et al., 2015) and RANSAC (Schnabel et al., 2007) to assess the ability of these methods for segmenting the surface point cloud, which is measured through quantitative evaluations (e.g. F1-score) (Truong-Hong and Laefer, 2015a) by comparing to ground truth (GT). Finally, in Issue 3, either contextual knowledge and design documentation or the fitting surface–based point clouds are used as the reference surface S_ref for computing deformation of the structure. Investigating this issue would provide a valuable guideline for practitioners in selecting the appropriate reference surface when it is not explicitly available. Moreover, a moving smooth method based on a spatial interval is introduced to the minimum impact of Issue 4. In the following sections, three case studies are used to demonstrate this investigation.

4.1 Lateral deformation of a masonry abutment

An abutment of a metal railway bridge in the Pegnitz Valley, Bayreuth district of Bavaria, Germany, is selected, which was made from block stones. The bridge built in the 1920s has a span of about 37 m. However, after nearly 100 years’ service, the abutment was subjected to movements, which can affect the safety of the bridge. As such, the deformation of the abutment components must be estimated to establish maintenance planning. A Leica P20 scanner with a scanning range from 0.4 m to 120 m, point accuracy of 3 and 6 mm for a range measurement of 50 and 100 m, and angular accuracy of 0.002 degree was used to collect point clouds of the abutment (Leica Geosystems, 2021). The scanner was set up about 8 m from the abutment, which allows capturing front and side surfaces of the abutment from a single scan station with a sampling step of 6.3 mm at 10 m (Figure 1a). Subsequently, irrelevant points (e.g. points of plants and a superstructure) were manually removed from the Leica Cyclone (Leica Geosystems, 2020), and the remaining point cloud was down-sampled with the sampling step by 10 mm before exporting their x-, y- and z-coordinates for deformation estimation (Figure 1b).

As the abutment was scanned from a single location, a registration procedure is not required during pre-processing, which implies that there is no registration error. This case study focuses on investigating Issue 2 and 3. For Issue 2, RANSAC (Schnabel et al., 2007) implemented in Cloudcompare (2021) and VRG (Vo et al., 2015) to extract point clouds of abutment components were used. For RANSAC, input parameters are no_ptc_min = 100,000 points, ε_max = 10 mm, α_max = 5.0 degrees, and β = 0.25. However, before extracting point clouds of surface, a normal vector of each point is estimated, which is normal of a plan fitted through its neighbouring points. In this study, a range search with a searching radius of 0.1 m is used to retrieve the neighbouring points for each given point. Moreover, for VRG, input parameters are empirically selected as follows: vs₀ = 0.25 m, α₀ = 5.0 degrees, d₀ = 10 mm and res₀ = 10 mm. Issue 2 is to investigate an appropriate segmentation method to extract the structure surfaces, and it is unnecessary to select the same input parameters for both methods. However, in this case study, they were selected nearly the same, for example, pairs of parameters, ε_max and d_v, and α_max and α_v, respectively, describe deviation of distances and normal between the points and the model, while and a pair of β and vs₀ is examined if the points can be considered to check if they are part of the model.

Results of the segmentation derived from RANSAC and VRG based on input point clouds in Figure 1b are shown in Figure 2. Both methods extract the points of the abutment components properly. However, RANSAC extracts the wing wall as two segments (Figure 2a), which implies that the wing wall is subjected to large deformation, and the primitive plane is not suitable to represent an entire wing wall. Moreover, the resulting segmentation based on RANSAC also excludes points around edges where two components (e.g. wing and ballast walls, and wing and breast walls) are intersected. That is because the normal vectors of those points are significantly different from a normal of the model (Figure 2b). This phenomenon does not occur in resulting segmentations derived from VRG because this method includes a filtering step to refine under- or over-segmentation around edges (Vo et al., 2015).

Resulting surface extraction shown in Figure 2 is evaluated by comparing them to GT manually extracted from the abutment (Figure 1b). A point-based evaluation strategy based on quantitative indicators including a true positive (TP), false positive (FP) and false negative (FN) were used to identify a difference between data points of the abutment components extracted from RANSAC and VRG and those of GT. Evaluation quantities are interpreted through completeness (Comp.), correctness (Corr.), and F1-score. For a definition of these quantities and the methodology to compute them, one can refer to Truong-Hong and Laefer (2015a).

The evaluation showed that both methods extracted the point clouds of the abutment components with high accuracy. Indeed, Comp., Corr., and F1-score were respectively no smaller than 94.3%, 96.6% and 0.969 for RANSAC, and 97.9%, 97.9% and 0.979 for VRG (see Table 1). Generally, VRG extracts the data point of surfaces of the abutment components better than RANSAC. For example, for the wing wall, F1-score from VRG is 0.049, larger than that from RANSAC.

For Issue 3 investigating the impact of a reference surface S_ref on deformation estimation, two types of the reference surface S_ref are used in this case study. First, the best fitting plane of the abutment components based on their point clouds is considered as the reference surface S_ref,fit, while second as-design surfaces of the abutment components are assumed as the reference surface S_ref,design. For the first case, a plane is fitted through the GT point clouds (Figure 1c) of the abutment components by using robust principal component analysis (rPCA) (Laefer and Truong-Hong, 2017) based on a weight covariance matrix cov_w (Equation 2). The fitting plane (S_fit = (p₀, n)) is defined by a point p₀ = (x₀, y₀, z₀) (Equation 3) and a normal vector n = (n_x, n_y, n_z), which is an eigenvector corresponding to the smallest eigenvalue. The process is iteratively to update the weight of each point to estimate the plane until the difference of fitting residual of two consecutive iterations is no larger than a predefined threshold, Δres₀ = 0.001, which is empirically selected (Equation 5).

(2)covw=∑pi∈Pw(pi)(pi−p0)(pi−p0)T∑pi∈Pw(pi)

(3)p0=∑pi∈Pw(pi)pi∑pi∈Pw(pi)

where p_i = {x_i, y_i, z_i} ∈ R³, and w(p_i) is the weight of the point expressed in Equation (5).

(4)w(pi)=exp(−di2do2)

where d_i is the Euclidean distance from the point p_i to the fitting surface, and d₀ is an average distance of d_i.

(5)Δr=|resi+1−resi|≤Δres0=0.001

where res_i and res_i + 1 are the residual of iteration i and i + 1, which is expressed in Equation (6).

(6)resi=1|di|∑i=1Ndi2

where d_i is Euclidean distance from the point p_i to the fitting plane S_fit = (p₀, n) (Equation 7).

(7)di=(xi−x0)nx+(yi−y0)ny+(zi−z0)nznx2+ny2+nz2 

For the second case, contextual knowledge derived from a design concept about the abutment components is used to determine their as-design surfaces. In the bridge design, the ballast and breast walls are perpendicular to the wing wall. As edges of the component connected to adjacent components are stiff, areas surrounding the edges are non-deformation. These areas can be selected to determine the surface representing an as-design status of the component. To get the surface determination from GT, points in the vicinity of the edges of a given component are extracted. The process starts with the points of the adjoined component to search the closest points within the given component, where the distances between the points of two surfaces are no larger than the predefined buffer (d_buffer). In this study, the buffer d_buffer = 0.2 m is empirically selected. Resulted extraction of the points of these areas is shown in Figure 3. Finally, rPCA (Equations 2–7) is employed to determine the as-design reference surface S_ref,design. Notably, for the breast wall, the bottom and top edges are also connected to other surfaces, while the right edge is not considered because a part of the breast wall was captured.

The resulting identification of the reference surface S_ref,fit and S_ref,design is shown in Table 2. A large root means square error (RMSE) of a fitting plane for the wing wall for the first case implies the wall is subjected to large deformation. Distances from the points of the abutment to the reference surface (S_ref,fit and S_ref,design) are computed (Equation 7). The resulting deformations are illustrated in Figures 4 and 5.

In each case, patterns of deformations based on RANSAC and VRG are nearly similar, while the maximum values are slightly different. For example, minimum and maximum deformations differ 1.26 mm (RANSAC vs VRG) and 1.87 mm (RANSAC vs VRG) for the reference surface S_ref,fit, and 5.74 mm (RANSAC vs VRG) and −2.52 mm (RANSAC vs VRG) for the reference surface S_ref,design (Figures 4 and 5, and Table 3). That implies that although there is different accuracy of surface extraction between RANSAC and VRG, differences of resulting deformations based on RANSAC and VRG are still minor. However, as RANSAC misses extracting points on the surface, the critical deformations are also excluded (Figures 4 and 5). Importantly, by using a fitting plane as the reference surface S_ref, the deformations of the abutment components appear as unreliable results (Figure 4). For example, there are large deformations at edges adjoined to the ballast and breast walls for the wing wall, which is unrealistic. Moreover, there is no deformation at the middle of the wing wall, while other areas are subjected to deformations.

On the other hand, the approach to determine the reference surface–based contextual knowledge gives realisable deformation. For the wing wall, deformations increase for the part away from the ballast and breast walls because the wing wall behaves similar to the cantilever beam, which is fixed at the edges connected to the ballast and breast walls (Figure 5). Moreover, there is no deformation around the edges of the breast wall, while small deformations are found away from the edges, which is due to lateral pressure load. Therefore, it can be concluded that the reference surface S_ref,fit based on a fitting plane, may not be used for deformation measurement, particularly when the surface is subjected to large deformation.

4.2 Deformation of the building slabs

An office building on Pham Ngu Lao st., Vietnam, was selected to be scanned by a Trimble TX8 with a maximum scanning range of from 0.6 m to 120 m (a range systematic error less than 2 mm) and an angular accuracy of 80 μrad in both vertical and horizontal (Trimble, 2020b). A point spacing of 11.3 mm at a range of 30 m and a total of ten scan stations were established to capture an interior storey with maximizing data coverage (Figure 6a). The point clouds were registered by the Trimble RealWork software v11.2 (Trimble, 2020a) with a registration error of about 1.57 mm. In this case, a part of the building ceiling is selected, which is only slabs over one span of a ground storey about 7.2 m wide and 7.2 m long, supported by the primary and secondary beams (Figure 6b). A subset includes 7.6 million data points of scanning stations 7, 8 and 9.

This case study investigates the impact of data quality due to registration (Issue 1), the quality of the surface extraction (Issue 2) on measuring vertical displacements of ceiling slabs and available outlier and noisy data (Issue 4). For Issue 1, data quality underlying a registration error is examined by comparing slab vertical displacements based on the point clouds from single and multiple scanning stations (SS and MS). Similar to case study 1, for Issue 2, RANSAC (Schnabel et al., 2007) and CRG (Truong-Hong and Lindenbergh, 2020) are employed to extract data points of the ceiling slabs. Moreover, for minimizing impact of outlier points available in resulting segmentation, a moving smooth method with a spatial interval is applied to vertical displacements of sections of the slabs (Issue 4).

To extract data points of the ceiling slabs, for RANSAC, input parameters include no_ptc_min = 5,000 points, ε_max = 5 mm, α_max = 2.5 degrees, and β = 0.5, while parameters for CRG are cs₀ = 0.5 m, α₀ = 2.5 degrees, d₀ = 5 mm and res₀ = 5 mm. These parameters are selected through several trials to get the best results. Both methods can extract the point clouds of the ceiling slabs properly (Figures 7a and b). Similar to case study 1, quantitative evaluation including Comp., Corr. and F1-score are used to interpret the quality of surface extraction from those methods comparing to GT (Figure 7c), which is manually extraction from the subset consisting of data points of scanning stations 7, 8 and 9 (Table 4). The point-based evaluation shows that CRG extracts data points of the ceiling slabs slightly better than RANSAC, where F1-score are 0.95 (RANSAC) and 0.99 (CRG) (Table 4).

For computing vertical displacements of ceiling slabs, the reference surface S_ref,i (i = [1;4]) of each ceiling slab (CS_i) is first determined. Based on contextual knowledge, the ceiling slab CS_i supported by primary and secondary beams are fixed at intersection edges between the ceiling slab and the beams. Aiming to determine these intersection edges, point clouds of the ceiling slab CS_i and of the vicinity are decomposed into 2D cells (C = {c_i, i = [1; N_c]}) in x and y directions, which is similar to the first step of CRG (Truong-Hong and Lindenbergh, 2020) (Figure 8a). Next, cells are classified into boundary cells (c_ext,i ∈ C) located on the slab boundaries and interior cells (c_int,j ∈ C). For each boundary cell c_ext,i, points (p_i ∈ c_ext,i) are separated into two groups: the slab points, p_cs,i and non-slab points p_ncs,i, where p_i = p_cs,i U p_ncs,i and p_cs,i ≠ ∅ (Figure 8a). Subsequently, a point-based region growing proposed by (Rabbani et al., 2006) was employed to segment the points p_ncs,i into a set of regions R = {R_j, j = [1;N_R]}. The region R_j = {p_j} is considered as a vertical surface of a supported beam of the ceiling slab CS_i if the fitting plane (S_j) of the region R_j are (1) perpendicular to the local surface s_cs,i of the ceiling slab and (2) the closest to the ceiling slab CS_i (Figure 8b). In this study, rPCA (Equations 2–7) is used to determine s_cs,i and S_j. Additionally, an intersection line (L_edge,i) between s_cs,i and S_j is determined, and the middle point of the intersection line segment is considered as an edge point p_edge,i (Figure 8b). Finally, the reference surface S_ref,i is a fitting plane though edge points (p_edge = {p_edge,i}) of ceiling slab CS_i by using rPCA (Equations 2–7). Resulting reference surfaces of the ceiling slabs are present in Table 5. For details of a procedure of extracting the reference surface S_ref,i, one can refer to (Truong-Hong and Lindenbergh, 2020). Importantly, GT point clouds of the ceiling slabs are used as input data to determine the edge points in this study.

A vertical displacement at a point p_i ∈ CS_i is defined as an orthogonal distance from the point p_i to the reference surface S_ref,i (Equation 7). Resulting vertical displacements based on the slab points derived from GT, RANSAC and CRG are shown in Figure 9, while statistical quantities of the vertical displacements of each ceiling slab are present in Table 6. Patterns of vertical displacements are similar (Figure 9), but the minimum and maximum vertical displacements are different (Table 6). Notably, the vertical displacements from CRG are highly consistent with those from GT, where averaged differences of absolute mean and mean are respectively −0.03 and 0.05 mm (Table 6), while these values between RANSAC and GT are 0.07 and 0.10 mm. However, there is a large difference in the maximum and minimum values between CRG and GT. For example, the maximum vertical displacements are 31.2 mm for GT and 18.9 mm for CRG. This implies that outlier points may still be available in GT.

In the building ceiling slab, large vertical displacements often occur at the middle spans, and vertical displacements along four sections are observed (Figure 9). As noisy data points are inevitable, a moving average method is employed to smooth vertical displacements, in which an interval space (w) of 0.05 m is used to compute the vertical displacement at the predefined location p_i along the section (Equation 8).

(8)δ‾vert,i=∑ δj|δj|

where δ_j is a vertical displacement at the data point p_j, | | denotes the number of points within the interval space w, which is determined based on Equation (9).

(9){pi.y−w2≤pj.y≤pi.y+w2 for Section 1 and 2pi.x−w2≤pj.x≤pi.x+w2 for Section 3 and 4

where suffixes “.x and.y” donate x- and y-coordinates of the points. Notably, the predefined locations p = {p_i} are linearly generated along the section with the interval space w = 0.05 m.

Results of vertical displacements along four sections are shown in Figures 10 and 11. Generally, deformation shapes from GT, RANSAC and CRG are similar, but outlier points are still available in GT (Figure 10a) and RANSAC (Figure 11c). However, CRG can eliminate outlier points and trim a few noisy data points. Finally, comparing the vertical displacements based on the point clouds derived from RANSAC and CRG against those based GT shows that results-based CRG highly agree with those from GT (Table 7). Although an average of the mean vertical displacements is 0.05 mm (RANSAC vs GT) and 0.02 mm (CRG vs GT), the averaged minimum/maximum vertical displacements are 1.05 mm/0.40 mm (RANSAC vs GT), and 0.84 mm/0.015 mm (CRG vs GT). Results prove that although the performance of RANSAC in extracting the points of the ceiling slabs is lower than that of CRG, it would not cause a significant difference in measuring the vertical displacements. However, the vertical displacements at cross-sections along sections vary about 10 mm (Figures 10a and c, 11a and c, and 12a and c), which reflects that the data quality is relatively low, which may be caused by a sensor and/or a registration error.

Addressing the impact of data quality on vertical displacement measurement (Issue 1), vertical displacements based on one scanning station (SS) point clouds are compared to those based on multiple scanning station (MS) ones. In this case study, the scanning station 8 below these ceiling slabs has a short scanning distance and small incidence angle compared to other stations, which is expected to give the best quality of the point clouds. As CRG is better than RANSAC in surface extraction from the MS subset, only CRG is used to extract the point clouds of the ceiling slab from the SS subset consisting of 3.7 million points, in which input parameters are similar to ones applying for the MS subset. Subsequently, reference surfaces (S_ref,i) for each ceiling slab (CS_i) are estimated from edge points extracted from the SS subset (Figure 13a and Table 8). The reference surfaces from the SS subset are identical to those based on the MS subset.

Finally, vertical displacements of the slabs are computed by using Equation (7), and results are illustrated in Figure 13b. The vertical displacements based on the SS subset highly agree with those based on the MS subset. This trend is also observed through the vertical displacements along four sections (Figure 14). Differences of mean vertical displacements between CRG-MS and CRG-SS are mostly less than 0.6 mm (Table 9), while differences of maximum vertical displacements are no larger than 1.6 mm (14.7 mm for CRG-MS vs 13.1 mm for CRG-SS in Slab 1 along Section 1). Moreover, average noise levels of MS and SS subsets are respectively 1.0 mm (std = 0.88 mm) and 1.0 mm (std = 0.86 mm) (Figure 15). The upper bound of the noise levels for the 95% confidence interval is 1.13 mm (the MS subset) and 1.16 mm (the SS subset) when using Weibull distribution. Thus, with a magnitude order of the noise level of the subsets, it can be arguably concluded that for this case, the noise would be strongly affected on vertical displacements, while a registration error of 1.57 mm causes a minor effect. This finding was also reported by Pesci et al. (2013). Notably, the noise is measured as the distance from the points to the fitting plane through its neighbouring points, which are 20 nearest neighbouring points used in this study. For details on computing noise, one can refer to Soudarissanane et al. (2011), Laefer et al. (2014).

4.3 Deformation of steel frame

As part of tank inspection (American Petroleum Institute, 2014), a steel frame supporting the tank roof must be assessed, in which deformation of the girders is a part of the assessment. In this case study, the oil tank located in Alberta in Canada, which is 27.4 m in diameter and 14.6 m in height, was used. The steel frame is located at an elevation about 15 m from the tank floor. Faro Focus X130 (Faro, 2020b) with a scanning range from 0.6 m to 130 m, a range error of ± 2 mm was used to capture the steel frame from an averaged distance of 15.0 m. Due to the occlusion, to obtain full coverage point clouds of the steel frame, a total of five scanning stations were set up, and they captured the structure with the sampling step by 6.136 mm at a measurement range of 10 m (Figure 16a). The point clouds from different scanning stations were registered by using Faro PointSense software version 18.5 (Faro, 2020a) with a mean point error of 0.9 mm and a maximum point error of 2.1 mm. As this study aims to demonstrate factors affecting the deformation measurement–based laser scanning point clouds, only data points of a girder of interest were selected (Figure 16b). The complete point cloud of the girder comes from three scanning stations 2, 4 and 5. Subsequently, the point clouds of the girder of interest and its adjoining structures (e.g. column heads) were manually extracted within Cloudcompare (2021).

Case study 2 shows that the quality (due to a registration error) of a point cloud would affect resulting vertical displacements. This case study demonstrates the effectiveness of the point cloud quality (Issue 1), selecting segmentation methods (Issue 2) and available outlier and noisy data (Issue 4). Moreover, an impact of mixed pixels (Wang et al., 2016) during capturing steel structures is also included in Issue 4. Two subsets (1) multiple scan stations (MS) (including stations 2, 4 and 5), and (2) a scan station 5 (SS) are used to measure vertical displacements of the girder (Figure 16b and c). The use of MS subsets reflects that in practice, the complete point cloud of a structural member is not sometimes obtained from one scanning station, while an SS subset can give insight impact of a registration error on deformation measurements. Notably, Station 5 is selected as its point cloud is given the best coverage of the girder.

Both RANSAC (Schnabel et al., 2007) and VRG (Vo et al., 2015) are employed to extract data points of the girder surfaces. As the vertical displacement of the girder is the main interest, the segmentation aims to extract a point cloud of the bottom surface (S_girder,bot) of the girder. For RANSAC, input parameters are min_ptc = 500 points, ε_max = 10 mm, α = 5.0 degrees and β = 0.2, while for VRG the parameters are vs₀ = 0.1 m, α₀ = 5.0 degrees, d₀ = 5 mm and res₀ = 5 mm. Notably, in RANSAC, a normal vector of each point is a normal plane fitted through 20 nearest neighbour points. Results of segmentation showed that RANSAC and VRG successfully extract data points of the bottom surface S_girder,bot (Figure 17). Moreover, a visualisation evaluation shows that under-segmentation occurs in S_girder,bot derived from the SS subset (Figure 17d and e). Quantitative evaluation by comparing points of S_girder,bot from RANSAC and VRG against GT are shown in Table 10. It is shown that for the MS subset, both methods extract S_girder,bot with higher accuracy from the MS subset than from the SS subset. For MS subset, Comp. and F-1 score are respectively no smaller than 98.9% and 0.991, while for the SS subset, they are no larger than 94.0% and 0.917 (Table 10). Interestingly, although VRG performs better than RANSAC for the MS subset, the opposite trend is found for the SS subset. This implies that a lower sampling step of the SS subset can affect the quality of a fitting plane within the voxel. Consequently, that would lead to under-segmentation.

As an initial surface or a reference surface, S_ref of the girder is not available, to compute vertical displacement, a surface through a set of points at two ends of the bottom surface S_girder,bot, is assumed to be the reference surface S_ref . This assumption is based on the reality that the girder was welded to the column, so two girder ends are being fixed. To do that, points within the buffer of 0.05 m from two ends of S_girder,bot are extracted (Figure 18), and rPCA (Equations 2–7) is employed to fit the plane. In this case study, point clouds of S_girder,bot from GT, estimate S_ref for the MS and SS subsets, and the results are shown in Table 11. There is a very slight difference between S_ref from two subsets, where the deviation between their normal is about 1 degree.

Additionally, vertical displacements of the girder are computed by Equation (7), which are orthogonal distances from points of S_girder,bot to the corresponding reference surface S_ref. Results of vertical displacements for each point are shown in Figure 19. Generally, the vertical displacements based on RANSAC and VRG highly agree with those based on GT for both subsets (Table 12). Differences of absolution mean and mean of vertical displacements are no more than 0.5 mm for both subsets. However, there are significant differences in the minimum and maximum vertical displacements based on RANSAC. For the minimum vertical displacements, they are −13.3 mm (−26.4 mm for RANSAC-MS vs −13.2 mm for GT-MS) and −16.6 mm (−28.2 mm for RANSAC-SS vs −11.6 mm for GT-SS) (Table 12). A similar trend is also found in the maximum vertical displacement from the SS subset with a difference of 21.1 mm (Figure 19 and Table 12). This implies that mixed pixels are still available around the edges of S_girder,bot derived from RANSAC (Figure 20). This phenomenon is eliminated chiefly from resulting point clouds from VRG.

Figure 21 shows vertical displacements at a middle section of S_girder,bot along a longitudinal direction of the girder and smooth vertical displacements by applying a moving average method presented in case study 2. Vertical displacements are nearly identical for all cases (Figure 21 and Table 13). Interestingly, differences of all statistical quantities are no larger than 0.4 mm for VRG vs GT with the SS subset. Moreover, there is a very slight difference between results-based MS and SS subsets, in which mean values of vertical displacements differ no more than 0.1 mm. Thus, once the mixed pixels are eliminated, the vertical displacements based on the MS and SS subsets are the same.

Similar to case study 2, noise levels of both subsets are similar, where the average noises are respectively 0.4 mm (std = 0.3 mm) for the MS subset and 0.4 mm (std = 0.3 mm) for the SS subset (Figure 22), while the upper bound of the noise levels corresponding to the 95% confidence interval are 1.1 mm (the MS subset) and 1.1 mm (the SS subset) when using Weibull distribution. That can translate with high accuracy of a registration (the average registration error of 0.9 mm) in this case study, resulting deformation estimation does not get affected. Thus, the use of point clouds from multiple scanning stations would give accurate deformation of a structure. Notably, the local plane at a given point for computing the noise is determined from 20 nearest neighbouring points.

5. Discussions

The paper presents an application of terrestrial laser scanning point clouds to measure deformation for structural assessment. The deformation is defined as a distance between a point cloud representing a current surface and a reference surface S_ref representing the initial or non-deformed surface of the structure. Structural components of a bridge, building and storage tank are used to demonstrate deformation measurement and investigate the impact of point cloud registration error, selected segmentation methods to extract a point cloud of a surface, selection of the reference surface S_ref, and available outlier, noisy data and/or mixed pixels.

Clearly, the reference surface plays a crucial role in computing the deformation. Case study 1 illustrates that the deformations of the abutment components are essentially different when different reference surfaces are applied. For example, the maximum lateral deformations of a wing wall increase from 54.36 mm to 254.15 mm for points of a surface derived from RANSAC when the reference surface based on a fitting plane (S_ref,fit) and a design concept (S_ref,design) are applied. Based on deformation distribution of the abutment components in case study 1, where unrealistic deformation appears in several areas of the surfaces, it can arguably conclude that the fitting plane based on the points of the surface may not be the reference surface when the structural surface is subjected to damage or large deformation. For example, RMSE of a fitting plane for the wing wall is 17.0 mm (Table 2). That is because the effects of point clouds of damage areas on a fitting plane cannot be eliminated although a robust fitting approach is applied. Therefore, S_ref,design would be recommended. However, due to data quality or imperfect construction, S_{ref, design} obtained from point clouds would differ from design documentations. For example, in principle, in the case study 2, points on slab edges should be coplanar, but results of fitting planes through their points show RMSE up to 5 mm (Tables 5 and 8). Moreover, as the complexity of a structure, the resulting segmentation based on RANSAC and CRG/VRG still includes data points of adjoined structures (Figure 21), and these points also impact estimating the reference surface.

Technically, RANSAC and CRG or VRG successfully extract point clouds of surfaces of structures. However, both are required to tune input parameters to obtain the best results (Vo et al., 2015). However, as RANSAC is a model-based method, the method is only suitable for the surface subjected to a small deformation, where the points on the deformed areas have distances to the model smaller than ε_max. When the surface is subjected to large deformation, RANSAC can fail to extract complete points of the surface, where parts of the large deformation area are excluded. For example, in case study 1, RANSAC (Schnabel et al., 2007) segments the point cloud of the wing wall into two clusters, and point clouds of some areas also exclude (Figure 2). Moreover, resulting in point clouds based on RANSAC can contain outlier points or points of adjoined structures (Figure 11a), which can cause extreme deformations to be reported. On the other hand, as region growing–based methods are applicable for extracting a free-form surface (Vo et al., 2015), they are suitable for the structure surfaces subjected to arbitrary deformation. In most cases, CRG and VRG give higher accuracy in extracting the points of the surface than RANSAC. However, VRG would have a low performance when a low density of the point cloud is used, as shown in case study 3 (Figure 19f). Furthermore, VRG is also better than RANSAC in eliminating mixed pixels surrounding the edges of a steel structure (Figure 19). Except for outlier points, deformation of the structure-based data points derived from RANSAC and CRG/VRG are similar to ones from GT. Thus, to give reliable deformations of the structure based on the point cloud, one of the key points is to minimize outlier points in the surface data points.

Mixed pixels, noise and a registration error of data points inevitably occur and are critical factors in measuring the deformation of a structure based on TLS data points. The registration error is completely removed when a point cloud of a single scanning station is used. However, that would not be generally applicable in practice because the complexity and size of the structure are restricted to obtain full coverage data of the structure from one scanning station. Thus, the registration error must be minimized. It can be seen that although point clouds from multiple scanning stations are used, the registration errors of 1.57 mm (case study 2) and 0.9 mm (case study 3) do not cause a negative impact on the resulting deformation.

Although CRG and VRG can partially reduce the availability of mixed pixels and noisy data points, point-based deformation can report extreme deformations at outmost points (either mixed pixels or outlier points), leading to an incorrect decision to be reported. Fortunately, as the mixed pixels mostly occur surrounding edges of the structure (Laefer et al., 2014), the points within a buffer of the surface edges would be excluded. For example, when the points of the middle sections are used, the effects of the mixed pixels are eliminated (Figure 21). Alternative outlier removal techniques, for example, statistical outlier removal proposed by Rusu et al. (2008), would be used to remove the mixed pixels. However, data points of damaged areas, which are subjected to large deformation, can also be removed because their spatial information may be similar to the mixed pixels or outlier points. In addition, an effect of noisy data, where the large deformation to be reported at outlier points (Jafari et al., 2017), can be reduced by using a soothing technique. For example, a moving average method is used in this study. A cell-to-cell method proposed by Truong-Hong and Lindenbergh (2019) can be an alternative method applied for a structural surface. Irrespective of smoothing techniques applied, the spatial interval (or a spacing interval) should be used rather than the number of points because of the uneven distribution of the point clouds. Moreover, the use of the spatial interval can cause discontinuity deformation between adjacent intervals, which can be solved by using multiple spatial intervals (Park et al., 2017).

One of the advantages of structural assessment-based TLS data is that status of an entire structure can report details. Results can be integrated into a BIM model allowing inspection, monitoring and assessment of an entire structure (Nguyen et al., 2020). Moreover, as data acquisition and processing cost is significantly reduced, the structure can be monitored and inspected with higher frequency, and structural deficiencies can be detected earlier. However, the accuracy of the scanning TLS scan is often about 1–2 mm (Faro, 2020b, Leica Geosystems, 2021), which still exceeds an allowable error in measuring the deformation of a specific structure. Moreover, a high accuracy of resulting point clouds is achieved when an optimal sampling step is used. Lichti and Jamtsho (2006) concluded that the optimal sampling step is equal to 86% of the width of the laser beam, which was based on analysing 11 commercial TLS scanners with a scanning range of about 50 m. Soudarissanane et al. (2011) and Laefer et al. (2009) reported that the orthogonal distance from the scanner to a surface is dominating accuracy of the point cloud, while a scanning angle should be less than 60 degrees. The total error budget should be considered as a critical key factor in selecting TLS to estimate deformation of a structure

6. Conclusions

Laser scanning, particularly TLS, captures surfaces of structures with high accuracy and low cost in operation. However, in practice, using TLS for structural assessment through deformation measurement has many challenges, mainly relating to the point cloud registration error, surface extraction and selection of a reference surface, and available outlier, noisy data and/or mixed pixels. This study illustrates the use of TLS data in estimating deformations for various types of structures, including a masonry bridge abutment, building slabs from a construction project and a steel girder supporting an oil tank roof. This work is to demonstrate a procedure to estimate deformation of the structure and explore the impact of these challenges on resulting deformation estimation of the structure.

The study shows that both RANSAC and region growing–based methods (CRG/VRG) can be extracted data points of surfaces, but RANSAC is only applicable for a primary primitive surface (e.g. a plane in this study) subjected to a small deformation (case study 2 and 3) and cannot eliminate mixed pixels. On the other hand, CRG and VRG impose a suitable method applied for deformed, free-form surfaces. Moreover, CRG and VRG are also better than RANSAC in eliminating mixed pixels and noisy data. That would consequently report reliable and accurate deformation of the structure.

In addition, in practice, a reference surface of a structure is mostly not available. The use of a fitting plane based on a point cloud of an existing surface would cause unrealistic and inaccurate deformation because outlier and data points of damaged areas affect an accuracy of the fitting plane. This study would recommend using the reference surface determined based on a design concept/specification. Finally, the quality of point clouds, which are mainly dominated by outlier, noisy data, mixed pixels and a registration error, is also a critical factor in obtaining reliable and accurate deformation. Smoothing methods can be reduced a negative impact on the data quality; however, a noise level of data and the registration error are still needed to minimize, and the mixed pixels must be priority filter out a data set before computing deformation.

Figures

Figure 1

Point clouds of an abutment

Figure 2

Point clouds of the abutment components from RANSAC and VRG

Figure 3

Extract areas used to determine the as-design surfaces of the abutment components

Figure 4

Deformations based S_ref,fit

Figure 5

Deformations based S_ref,design

Figure 6

Point clouds of a building and areas of interest

Figure 7

Point clouds of ceiling slabs from RANSAC, CRG and GT overlaid input data

Figure 8

A procedure of edge point extraction

Figure 9

Vertical displacements of the ceiling slabs derived from GT, RANSAC and CRG

Figure 9

Figure 10

Vertical displacements along sections-based GT

Figure 11

Vertical displacements along sections-based RANSAC

Figure 12

Vertical displacements along sections-based CRG

Figure 13

Point clouds and vertical displacements from a scan station 8

Figure 14

Averaged vertical displacements along four sections from the SS subset

Figure 15

Noise levels of MS and SS subsets

Figure 16

Point clouds of a steel frame and a girder of interest

Figure 17

Point clouds of surfaces of the girder from various methods

Figure 18

Illustrated points within a buffer from the end of S_grider,bot for estimating S_ref

Figure 19

Vertical displacements of the girder from MS and SS subsets

Figure 20

Illustrated mixed pixels available in S_girder,bot derived from RANSAC

Figure 21

Vertical displacements of a middle section along the girder

Figure 22

Noise levels of MS and SS subsets

Table 1

Summarized comparison of surface extraction based RANSAC and VRG vs GT

Component	TP	FP	FN	Comp	Corr	F1-score
RANSAC vs GT(*)
Ballast wall	413,423	1,184	22,934	94.7%	99.7%	0.972
Breast wall	517,487	32	28,294	94.8%	100.0%	0.973
Wing wall	3,383,824	13,880	204,577	94.3%	99.6%	0.969
VRG vs GT
Ballast wall	428,145	1,667	8,212	98.1%	99.6%	0.989
Breast wall	534,466	11,285	11,315	97.9%	97.9%	0.979
Wing wall	3,579,463	31,574	8,938	99.8%	99.1%	0.994

Note(s): ^(*) Point clouds of Segment 1 and 4 (Figure 2a) are considered as the points of the wing wall for evaluation

Table 2

Parameters of the reference surface

Component	p0 = (x0, y0, z0)	n = (nx, ny, nz)	RMSE (mm)
Sref,fit
Ballast wall	(9.956, 0.681, 6.734)	(−0.141, −0.99, −0.008)	3.6
Breast wall	(11.62, −0.791, 1.322)	(−0.146, −0.989, 0.017)	8.0
Winging wall	(9.005, 3.248, 4.941)	(−0.991, 0.128, 0.025)	17.0
Sref,design
Ballast wall	(8.880, 0.835, 6.191)	(−0.128, −0.992, −0.006)	2.7
Breast wall	(9.404, −0.459, 1.209)	(−0.148, −0.989, 0.018)	8.2
Winging wall	(8.626, 0.414, 3.783)	(−0.994, 0.103, 0.031)	3.8

Table 3

Statistical quantities of deformations from different reference surfaces (unit = mm)

Component	Sref,fit					Sref,design
	Abs. mean	Mean	Std	Min	Max	Abs. mean	Mean	Std	Min	Max
Surfaces derived from RANSAC
Ballast wall	2.8	−0.8	3.4	−21.5	28.7	10.7	9.8	10.2	−13.4	34.0
Breast wall	5.9	−2.9	7.0	−25.9	27.4	6.9	−0.9	9.0	−38.5	30.0
Wing wall	14.1	0.6	16.8	−54.5	68.1	54.3	−53.9	55.0	−253.9	27.5
Surfaces derived from VRG
Ballast wall	2.9	−0.7	3.5	−38.0	27.9	10.3	9.2	10.2	−37.2	34.0
Breast wall	6.2	−3.0	7.4	−28.4	38.0	7.0	−0.8	9.0	−39.4	37.4
Wing wall	14.3	0.0	17.0	−55.8	66.3	53.8	−53.3	55.9	−259.6	34.0

Table 4

Evaluation of RANSAC and CRG

Quantitative	TP	FP	FN	Comp	Corr	F1-score
RANSAC vs GT	1,713,578	4,033	188,980	90.1%	99.8%	0.95
CRG vs GT	1,876,199	13,878	26,359	98.6%	99.3%	0.99

Table 5

Parameters of reference surfaces S_ref,i = (p₀, n) of the ceiling slabs

Slab	p0 = (x0, y0, z0)	n = (nx, ny, nz)	RMSE (mm)
Slab 1	(9.868, 13.231, 1.825)	(−0.007, 0.005, 1.000)	5.0
Slab 2	(9.868, 13.231, 1.825)	(−0.007, 0.005, 1.000)	3.0
Slab 3	(6.831, 14.985, 1.831)	(−0.006, 0.003, −1.000)	4.0
Slab 4	(8.130, 10.308, 1.835)	(0.004, 0.004, 1.000)	3.0

Table 6

Statistical quantities of vertical displacements from GT, RANSAC and CRG (unit = mm)

Component	Abs. Mean	Mean	Std	Min	Max
GT
Slab 1	4.8	−3.3	4.9	−18.0	23.6
Slab 2	5.2	4.9	3.7	−11.6	31.2
Slab 3	2.8	−0.2	3.3	−11.8	13.7
Slab 4	2.8	2.5	2.0	−8.2	13.8
RANSAC
Slab 1	4.9	−4.0	4.3	−22.1	8.0
Slab 2	5.4	5.2	3.5	−7.0	25.2
Slab 3	2.8	−0.3	3.4	−10.0	17.3
Slab 4	2.9	2.7	2.0	−7.3	19.3
CRG
Slab 1	4.7	−3.2	4.8	−15.1	18.9
Slab 2	5.2	4.9	3.6	−8.7	14.1
Slab 3	2.8	−0.2	3.4	−10.6	15.7
Slab 4	2.8	2.5	2.0	−7.8	13.9

Table 7

Statistical quantities of vertical displacements (unit: mm)

Methods	Mean	Std	Min	Max	Mean	Std	Min	Max
	Section 1 - Slab 1				Section 1 - Slab 3
GT	−5.0	5.4	−10.8	9.6	−0.5	3.1	−4.1	6.5
RANSAC	−5.2	5.0	−10.8	6.4	−0.4	3.2	−4.1	6.5
CRG	−4.9	5.4	−10.6	9.4	−0.5	3.1	−4.1	6.5
RANSAC vs GT	−0.2	0.6	−3.9	0.0	0.0	0.3	−1.0	2.4
CRG vs GT	0.1	0.2	−0.2	1.5	0.0	0.0	−0.3	0.1
	Section 2 - Slab 2				Section 2 - Slab 4
GT	5.2	3.9	−1.9	10.1	3.3	0.7	2.1	4.4
RANSAC	5.2	3.8	−1.9	10.1	3.3	0.7	2.1	4.4
CRG	5.2	3.8	−1.4	10.1	3.3	0.7	2.1	4.4
RANSAC vs GT	0.0	0.1	0.0	0.5	0.0	0.0	−0.3	0.1
CRG vs GT	0.0	0.1	0.0	0.4	0.0	0.0	0.0	0.0
	Section 3 - Slab 1				Section 3 - Slab 2
GT	−5.9	2.4	−10.9	−2.7	6.1	2.3	0.4	8.7
RANSAC	−5.9	2.4	−10.9	−2.6	6.1	2.3	0.4	8.7
CRG	−5.9	2.4	−10.6	−2.3	6.1	2.3	0.4	8.7
RANSAC vs GT	0.0	0.1	−0.4	0.1	0.0	0.0	−0.1	0.0
CRG vs GT	0.0	0.2	−0.5	0.6	0.0	0.0	0.0	0.0
	Section 4 - Slab 3				Section 4 - Slab 4
GT	−2.0	1.6	−4.5	1.0	2.3	0.8	0.6	3.6
RANSAC	−2.0	1.6	−4.5	1.3	2.3	0.8	0.6	3.6
CRG	−2.0	1.6	−4.5	1.0	2.3	0.8	0.6	3.6
RANSAC vs GT	0.0	0.0	0.0	0.4	0.0	0.1	0.0	0.4
CRG vs GT	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0

Table 8

Parameters of reference surfaces (S_ref = (p₀, n)) of the ceiling slabs

Slab	p0 = (x0, y0, z0)	n = (nx, ny, nz)	RMSE (mm)
Slab 1	(9.802, 13.237, 1.824)	(−0.007, 0.006, 1.000)	5.0
Slab 2	(7.795, 10.345, 1.837)	(−0.004, −0.004, −1.000)	3.0
Slab 3	(6.796, 15.049, 1.832)	(−0.006, 0.003, −1.000)	5.0
Slab 4	(5.006, 12.118, 1.840)	(−0.003, −0.003, −1.000)	3.0

Table 9

Statistical quantities of vertical displacements along sections from the SS subset (unit = mm)^(*)

Methods	Mean	Std	Min	Max	Mean	Std	Min	Max
	Section 1 - Slab 1				Section 1 - Slab 3
CRG-SS	−3.3	6.2	−9.9	13.1	−0.3	3.4	−4.4	6.5
CRG-S vs CRG-MS	0.3	0.8	−1.8	1.6	−0.1	0.3	−0.7	1.2
	Section 2 - Slab 2				Section 2 - Slab 4
CRG-SS	4.4	4.4	−3.6	10.1	3.9	0.8	1.7	5.7
CRG-S vs CRG-MS	−0.4	0.5	−1.5	0.4	0.6	0.5	−0.1	2.5
	Section 3 - Slab 1				Section 3 - Slab 2
CRG-SS	−5.3	2.5	−10.3	−1.1	5.6	2.5	0.4	8.2
CRG-S vs CRG-MS	0.4	0.2	0.2	1.2	−0.4	0.4	−1.3	0.3
	Section 4 - Slab 3				Section 4 - Slab 4
CRG-SS	−2.0	1.7	−4.7	1.3	2.8	0.8	0.5	4.2
CRG-S vs CRG-MS	0.0	0.3	−0.6	0.6	0.6	0.2	0.1	1.1

Note(s): CRG-MS is CRG presented in Table 7

Table 10

Evaluation of RANSAC and VRG for MS and SS subsets

Component	TP	FP	FN	Comp	Corr	F1-score
GT vs RANSAC-MS	19,134	125	215	98.9%	99.4%	0.991
GT vs VRG-MS	19,214	91	135	99.3%	99.5%	0.994
GT vs RANSAC-SS	1,755	204	112	94.0%	89.6%	0.917
GT vs VRG-SS	1,533	76	334	82.1%	95.3%	0.882

Table 11

Parameters of reference surfaces [S_ref = (p₀, n)] of the bottom surface of the girder

Data	p0 = (x0, y0, z0)	n = (nx, ny, nz)	RMSE (mm)
GT-MS	(2.294, 0.093, 14.190)	(−0.005, 0.01, 1.000)	1.0
GT-SS	(6.554, 0.083, 14.211)	(−0.005, 0.028, 1.000)	1.0

Table 12

Statistical quantities of vertical displacements from GT, RANSAC and VRG (unit = mm)

Methods	Abs. Mean	Mean	Std	Min	Max	Abs. Mean	Mean	Std	Min	Max
	MS subset					SS subset
GT	6.5	−6.5	3.1	−13.2	2.3	6.3	−6.2	3.3	−11.6	2.1
RANSAC	6.5	−6.5	3.2	−26.4	3.0	6.6	−6.2	4.3	−28.2	23.2
VRG	6.5	−6.5	3.1	−15.0	3.7	6.4	−6.4	3.3	−14.2	2.8

Table 13

Statistical quantities of vertical displacements from GT, RANSAC and CRG (unit = mm)

Methods	Mean	Std	Min	Max	Mean	Std	Min	Max
	MS subset				SS subset
GT	−6.3	3.0	−10.4	0.6	−6.4	3.1	−10.2	1.8
RANSAC	−6.3	3.0	−10.4	0.4	−6.4	3.1	−10.2	1.8
VRG	−6.3	3.0	−10.4	0.2	−6.4	3.1	−10.2	1.8
RANSAC vs GT	0.0	0.0	−0.1	0.0	0.0	0.0	0.0	0.0
VRG vs GT	0.0	0.0	−0.7	0.0	0.0	0.0	−0.1	0.4