Point Cloud Processing
Introduction
Point clouds are data points defined in a three-dimensional space, often generated by 3D scanners or LiDAR sensors. These data points represent the external surfaces of objects and scenes, making point clouds crucial for various applications such as 3D modeling, object recognition, and environment mapping. In this blog, we’ll explore how to work with point clouds using the Open3D library, covering essential tasks like reading, visualizing, downsampling, clustering, segmentation, and computing normal.
Open3D is an open-source library designed to provide an efficient and comprehensive platform for 3D data processing. It is widely used in academia and industry for various tasks involving 3D point clouds, meshes, and other 3D data structures. Open3D supports a range of operations, from basic 3D data handling to advanced tasks such as registration, reconstruction, and visualization. It is particularly popular for robotics, computer vision, and graphics tasks, where 3D data is increasingly critical.
One of the standout features of Open3D is its simplicity, ease of use, and powerful capabilities. The library offers a high-level Python API that makes it accessible to many users. Under the hood, Open3D is implemented in C++ to ensure high performance, particularly for large-scale 3D data processing tasks. Open3D supports 3D data formats, including point clouds, triangle meshes, and voxel grids. It also provides tools for 3D visualization, which are essential for exploring and understanding complex 3D data. With functionalities such as point cloud filtering, registration (alignment of 3D datasets), and surface reconstruction, Open3D is a versatile tool for a wide array of 3D applications.
In addition to its core features, Open3D constantly evolves, with an active community contributing to its development. This ensures that the library stays at the forefront of 3D data processing technology, integrating new algorithms and techniques as they emerge. Whether you are developing a robotics application, building a 3D scanning system, or conducting research in 3D computer vision, Open3D provides the tools you need to handle and process 3D data efficiently. Its ease of use, performance, and comprehensive functionality make Open3D an essential library for anyone working with 3D data.
1. Point Cloud Storage Formats
How data is stored can significantly impact your workflow when working with point clouds, especially when dealing with large datasets. Point clouds are typically stored in Binary and ASCII formats.
Binary Format
- The binary format is highly efficient for storing large datasets. It stores data compactly, reducing file size and speeding up the reading and writing processes. This is crucial when working with large point clouds, such as those generated by LiDAR sensors.
- Since binary formats store data in its raw numerical form, no precision loss can occur with text-based formats. This is particularly important for scientific and engineering applications where accuracy is paramount.
- Common binary formats include .ply in binary mode and .pcd in binary mode.
ASCII Format
- Unlike binary files, ASCII files store data as human-readable text. Each point in the cloud is represented by its coordinates (x, y, z) and possibly additional information like intensity or color. This makes it easy to inspect and modify the data manually.
- Although the ASCII format is less efficient than binary, it offers more flexibility for editing, debugging, and small-scale experiments where file size and speed are less critical.
Understanding the storage format is essential for selecting the right tools and methods for point cloud processing. Binary formats are more efficient and precise, making them suitable for large datasets, while ASCII formats are easier to work with when you need to inspect or edit the data manually.
Understanding Point Cloud Attributes
Point clouds can contain more information than spatial coordinates (x, y, z). Depending on the source and application, point clouds may include several other attributes:
- X, Y, Z: These are the primary coordinates representing the position of each point in 3D space.
- Intensity: Often used in LiDAR data, intensity represents the return strength of the laser signal. It can help distinguish between different types of surfaces, such as differentiating between asphalt and grass.
- Color (R, G, B): Points can also carry color information, adding richness to the visualization and enabling color-based segmentation or analysis.
- Normals: Normals are vectors perpendicular to the surface at each point, essential for tasks like rendering, surface reconstruction, and understanding the object’s geometry.
2. Point Cloud Processing Using Open3D
Open3D is a powerful library that provides many tools for working with point clouds. Let’s explore some common point cloud processing tasks using Open3D, with detailed explanations to help you understand the purpose and implementation of each step.
Install
1
pip install open3d
2.1. Creating a Point Cloud
Point clouds can be created from scratch, as arrays, or using existing data structures like tensors or numpy arrays. This section covers the basic methods for initializing and manipulating point clouds in Open3D.
Creating an Empty Point Cloud
Creating an empty point cloud is the first step when you want to start adding points programmatically.
1
2
3
4
import open3d as o3d
# Create an empty point cloud
pcd = o3d.t.geometry.PointCloud()
print(pcd)
The o3d.t.geometry.PointCloud() function initializes an empty point cloud object. This object can later be populated with 3D points, making it the starting point for various data processing tasks.
Creating a Point Cloud from Arrays
Point clouds can be directly created from numpy arrays or tensors, which allows for flexibility in how the data is input.
1
2
3
4
5
6
7
8
import open3d.core as o3c
import numpy as np
# Create a point cloud from a NumPy array
pcd = o3d.t.geometry.PointCloud(np.array([[0, 0, 0], [1, 1, 1]], dtype=np.float32))
print(pcd)
# Create a point cloud from a tensor
pcd = o3d.t.geometry.PointCloud(o3c.Tensor([[0, 0, 0], [1, 1, 1]], o3c.float32))
print(pcd)
Open3D allows you to initialize point clouds directly from arrays or tensors. This is useful when you already have your point data in another format, such as numpy arrays, commonly used in scientific computing. The flexibility of using different data formats like numpy arrays or tensors makes Open3D a versatile tool for integrating with other libraries.
2.2. Visualizing Point Clouds
Visualization is crucial for understanding and debugging your point cloud data. Open3D provides functions to render point clouds interactively.
1
2
3
4
5
6
7
8
# Load a PLY point cloud and visualize it
ply_point_cloud = o3d.data.PLYPointCloud()
pcd = o3d.t.io.read_point_cloud(ply_point_cloud.path)
o3d.visualization.draw_geometries([pcd.to_legacy()],
zoom=0.3412,
front=[0.4257, -0.2125, -0.8795],
lookat=[2.6172, 2.0475, 1.532],
up=[-0.0694, -0.9768, 0.2024])
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
-- Mouse view control --
Left button + drag : Rotate.
Ctrl + left button + drag : Translate.
Wheel button + drag : Translate.
Shift + left button + drag : Roll.
Wheel : Zoom in/out.
-- Keyboard view control --
[/] : Increase/decrease field of view.
R : Reset view point.
Ctrl/Cmd + C : Copy current view status into the clipboard.
Ctrl/Cmd + V : Paste view status from clipboard.
-- General control --
Q, Esc : Exit window.
H : Print help message.
P, PrtScn : Take a screen capture.
D : Take a depth capture.
O : Take a capture of current rendering settings.
to_legacy() converts the tensor-based point cloud into a format compatible with the legacy Open3D functions, such as draw_geometries.
The zoom parameter controls the camera’s zoom level in the visualization. A value of 0.3412 indicates a specific zoom level. Lower values zoom out, showing a broader view of the scene, while higher values zoom in, focusing on a smaller area.
The front parameter sets the camera’s view direction. The vector [0.4257, -0.2125, -0.8795] represents the direction from which the camera looks at the object. This vector is a normalized direction vector in 3D space. Changing this vector alters the angle at which you view the geometry in the 3D scene.
The lookat parameter specifies the point in 3D space that the camera is focused on. The point [2.6172, 2.0475, 1.532] is the exact location in 3D space that the camera is pointed toward. Adjusting this value changes the camera’s focal point, effectively shifting what the camera is centered on in the scene.
The up parameter defines the camera’s “up” direction, which determines how the scene is oriented vertically. The vector [-0.0694, -0.9768, 0.2024] represents the upward direction relative to the camera’s current orientation. Changing this vector can rotate the scene around the camera’s line of sight, effectively tilting the view.
The draw_geometries function opens an interactive window where you can rotate, zoom, and inspect the point cloud from different perspectives. Visualization is essential for checking the quality of your data before and after processing.
Note
o3d.t.io: This module is part of the Open3D tensor-based API, which is designed for modern data pipelines involving machine learning and large-scale processing. It supports operations that can leverage GPU acceleration and is intended for more advanced use cases.
o3d.io: This module is part of the legacy API, which is simpler and suitable for general-purpose point cloud processing. It is widely used for tasks that do not require the advanced capabilities of the tensor-based API.
Use o3d.t.io when you need to work with large datasets or integrate with machine learning workflows, and use o3d.io for simpler, more straightforward tasks.
2.3. Downsampling Point Clouds
Downsampling reduces the number of points in the cloud, making the data more manageable for processing. This is particularly important when working with large datasets.
Why Downsampling?
Performance: Large point clouds with millions of points can be computationally expensive to process. Downsampling reduces the number of points, speeding up operations like rendering, clustering, and segmentation.
Memory Usage: By reducing the point count, downsampling also decreases the memory required to store and manipulate the point cloud, which is crucial in resource-constrained environments.
Data Simplification: Downsampling can simplify the data while preserving its overall structure, making it easier to analyze and work with.
Voxel Downsampling
Voxel downsampling groups points into voxels (3D grid cells) and replaces all points within a voxel with a single representative point, effectively reducing the resolution of the point cloud.
1
2
3
4
5
6
7
# Downsample the point cloud with a voxel size of 0.03
downpcd = pcd.voxel_down_sample(voxel_size=0.03)
o3d.visualization.draw_geometries([downpcd.to_legacy()],
zoom=0.3412,
front=[0.4257, -0.2125, -0.8795],
lookat=[2.6172, 2.0475, 1.532],
up=[-0.0694, -0.9768, 0.2024])
This method simplifies the point cloud by averaging the points within each voxel. The voxel_down_sample function reduces the number of points, which speeds up subsequent processing steps without significantly compromising the overall structure of the point cloud.
Farthest Point Downsampling
This method selects points iteratively, ensuring that the selected points are as far apart as possible.
1
2
3
4
5
6
7
# Downsample the point cloud by selecting 5000 farthest points
downpcd_farthest = pcd.farthest_point_down_sample(5000)
o3d.visualization.draw_geometries([downpcd_farthest.to_legacy()],
zoom=0.3412,
front=[0.4257, -0.2125, -0.8795],
lookat=[2.6172, 2.0475, 1.532],
up=[-0.0694, -0.9768, 0.2024])
This technique is useful when retaining the most spatially distinct points in the point cloud. By selecting points that are maximally distant from each other, this method preserves the overall shape and structure of the data.
2.4. Normal Estimation
Normal estimation is essential for tasks like surface reconstruction and rendering, where understanding the orientation of surfaces is crucial.
Why Compute Normals?
Normals provide information about the orientation of surfaces in the point cloud, which is critical for tasks like surface reconstruction and rendering.
In rendering applications, normals determine how light interacts with surfaces, affecting the appearance of shading and reflections.
Normals can be used to identify features in the point cloud, such as edges or corners, which are important for tasks like object recognition and segmentation.
1
2
3
4
5
6
7
8
# Estimate normals
downpcd.estimate_normals(max_nn=30, radius=0.1)
o3d.visualization.draw_geometries([downpcd.to_legacy()],
zoom=0.3412,
front=[0.4257, -0.2125, -0.8795],
lookat=[2.6172, 2.0475, 1.532],
up=[-0.0694, -0.9768, 0.2024],
point_show_normal=True)
The estimate_normals function computes the surface normals for each point by analyzing the local neighborhood of points. Normals are vectors perpendicular to the surface and are critical for rendering, as they influence how light interacts with the surface.
Note
Access estimated normals
1
2
3
4
5
6
normals = downpcd.point.normals
print("Print first 5 normals of the downsampled point cloud.")
print(normals[:5], "\n")
print("Convert normals tensor into numpy array.")
normals_np = normals.numpy()
print(normals_np[:5])
2.5. Clustering Point Clouds
Clustering groups points that are close together into clusters can be useful for segmentation and object detection.
Why Clustering?
Segmentation: Clustering helps segment the point cloud into distinct objects or regions, which is crucial for tasks like object detection, recognition, and classification.
Noise Reduction: Clustering algorithms can help identify and remove noise from the point cloud by grouping points into clusters, improving the quality of the data.
Data Simplification: Clustering simplifies the analysis by breaking down the point cloud into smaller, more manageable groups that can be analyzed separately.
DBSCAN Clustering
DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is a popular clustering algorithm that identifies clusters based on the density of points.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
# Apply DBSCAN clustering
labels = pcd.cluster_dbscan(eps=0.02, min_points=10, print_progress=True)
max_label = labels.max().item()
colors = plt.get_cmap("tab20")(labels.numpy() / (max_label if max_label > 0 else 1))
colors = o3c.Tensor(colors[:, :3], o3c.float32)
colors[labels < 0] = 0
pcd.point.colors = colors
# Visualize the clusters
o3d.visualization.draw_geometries([pcd.to_legacy()],
zoom=0.455,
front=[-0.4999, -0.1659, -0.8499],
lookat=[2.1813, 2.0619, 2.0999],
up=[0.1204, -0.9852, 0.1215])
DBSCAN is a powerful clustering algorithm that groups points based on their density. It is particularly effective for detecting clusters of varying shapes and sizes and can also identify noise points that don’t belong to any cluster. In the visualization, different colors represent different clusters, helping you visually differentiate between them.
K-Means
The K-Means algorithm partitions the point cloud into clusters based on the Euclidean distance between points. Each cluster is represented by a centroid, and points are assigned to the nearest centroid. This method is effective for finding spherical or evenly distributed clusters in the dataset.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
from sklearn.cluster import KMeans
# Extract point positions (tensor) and convert to a 2D numpy array for clustering
points = np.asarray(pcd.point.positions.cpu().numpy())
# Apply K-Means clustering
# Apply K-Means clustering with explicit n_init parameter
kmeans = KMeans(n_clusters=5, random_state=42, n_init=10).fit(points)
labels_kmeans = kmeans.labels_
# Visualize the K-Means clustering result by assigning colors to each cluster
colors_kmeans = plt.get_cmap("tab10")(labels_kmeans / (labels_kmeans.max() if labels_kmeans.max() > 0 else 1))
pcd.point.colors = o3d.core.Tensor(colors_kmeans[:, :3], o3d.core.float32)
# Convert to legacy point cloud for visualization
pcd_legacy = pcd.to_legacy()
# Save and visualize the K-Means clustering result
o3d.io.write_point_cloud("kmeans_playground.ply", pcd_legacy)
o3d.visualization.draw_geometries([pcd_legacy], window_name="K-Means Clustering")
2.6. Segmentation of Planes
Plane segmentation identifies and isolates planar surfaces within a point cloud, such as walls or floors.
Why Segment Planes?
Object Detection: In indoor environments, planar surfaces like walls, floors, and tables are common. Segmenting these planes helps detect and isolate objects that rest on them.
Environmental Understanding: In robotics and autonomous systems, identifying planar surfaces is crucial for navigation and mapping.
Data Simplification: By segmenting out the planes, you can focus on processing the non-planar parts of the point cloud, which might represent objects of interest.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# Plane segmentation using RANSAC
plane_model, inliers = pcd.segment_plane(distance_threshold=0.01,
ransac_n=3,
num_iterations=1000)
[a, b, c, d] = plane_model.numpy().tolist()
print(f"Plane equation: {a:.2f}x + {b:.2f}y + {c:.2f}z + {d:.2f} = 0")
# Visualize the segmented plane
inlier_cloud = pcd.select_by_index(inliers)
inlier_cloud = inlier_cloud.paint_uniform_color([1.0, 0, 0])
outlier_cloud = pcd.select_by_index(inliers, invert=True)
o3d.visualization.draw_geometries([inlier_cloud.to_legacy(), outlier_cloud.to_legacy()],
zoom=0.8,
front=[-0.4999, -0.1659, -0.8499],
lookat=[2.1813, 2.0619, 2.0999],
up=[0.1204, -0.9852, 0.1215])
This technique uses RANSAC (Random Sample Consensus) to detect and isolate planar surfaces. The algorithm fits a plane to the points and iteratively improves the fit by discarding points that don’t conform to the plane model. This is particularly useful in applications like indoor mapping, where detecting walls, floors, and ceilings is essential.
The ransac_n parameter specifies the number of points that are sampled to estimate the plane in each iteration of the RANSAC algorithm. A value of 3 is typically used because three points are the minimum required to define a plane in 3D space. This parameter controls how the RANSAC algorithm generates plane hypotheses. Since three points define a plane, the algorithm randomly selects three points to propose a plane model in each iteration.
The num_iterations parameter specifies the number of iterations the RANSAC algorithm will run to find the best-fitting plane. A value of 1000 means that the algorithm will attempt to find the best plane by testing 1000 sets of three points. More iterations increase the likelihood of finding the best plane that fits the most inliers, but also increase the computation time. This is a balance between accuracy and performance.
2.7. Removing Outliers
Point clouds often contain noise, which can be removed using outlier removal techniques. Cleaning up the data improves the quality of further processing.
Why Remove Outliers?
Data Quality: Outliers often represent noise or errors in the data. Removing them improves the overall quality of the point cloud, making subsequent analyses more accurate.
Processing Efficiency: By removing unnecessary points, you reduce the size of the point cloud, making processing faster and more efficient.
Accuracy: Outliers can distort the results of algorithms like clustering, segmentation, and normal estimation. Removing them ensures that these processes yield more reliable outcomes.
Statistical Outlier Removal
This method removes points that are far from their neighbors based on a statistical analysis.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# Statistical outlier removal
# Downsample the point cloud with a voxel size of 0.03
voxel_down_pcd = pcd.voxel_down_sample(voxel_size=0.03)
# Remove statistical outliers
_, ind = voxel_down_pcd.remove_statistical_outliers(nb_neighbors=20, std_ratio=2.0)
# Convert the boolean mask to indices
ind = np.asarray(ind).nonzero()[0]
# Select inliers and outliers
inlier_cloud = voxel_down_pcd.select_by_index(ind)
outlier_cloud = voxel_down_pcd.select_by_index(ind, invert=True)
# Convert to legacy point cloud for visualization
inlier_cloud_legacy = inlier_cloud.to_legacy()
outlier_cloud_legacy = outlier_cloud.to_legacy()
# Visualize the inliers and outliers
print("Showing inliers (white) and outliers (red):")
inlier_cloud_legacy.paint_uniform_color([1, 1, 1])
outlier_cloud_legacy.paint_uniform_color([1, 0, 0])
o3d.visualization.draw_geometries([inlier_cloud_legacy, outlier_cloud_legacy],
zoom=0.8,
front=[-0.4999, -0.1659, -0.8499],
lookat=[2.1813, 2.0619, 2.0999],
up=[0.1204, -0.9852, 0.1215])
This method filters out points considered statistical outliers based on their distance to neighboring points. The remove_statistical_outliers function calculates the mean distance of each point to its neighbors and removes those that deviate significantly from the mean, thus reducing noise in the point cloud.
Radius Outlier Removal
This method removes points that do not have a sufficient number of neighbors within a given radius.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
#Radius Outlier Removal
# Remove statistical outliers
_, ind = voxel_down_pcd.remove_radius_outliers(nb_points=16,
search_radius=0.05)
# Convert the boolean mask to indices
ind = np.asarray(ind).nonzero()[0]
# Select inliers and outliers
inlier_cloud = voxel_down_pcd.select_by_index(ind)
outlier_cloud = voxel_down_pcd.select_by_index(ind, invert=True)
# Convert to legacy point cloud for visualization
inlier_cloud_legacy = inlier_cloud.to_legacy()
outlier_cloud_legacy = outlier_cloud.to_legacy()
# Visualize the inliers and outliers
print("Showing inliers (white) and outliers (red):")
inlier_cloud_legacy.paint_uniform_color([1, 1, 1])
outlier_cloud_legacy.paint_uniform_color([1, 0, 0])
o3d.visualization.draw_geometries([inlier_cloud_legacy, outlier_cloud_legacy],
zoom=0.8,
front=[-0.4999, -0.1659, -0.8499],
lookat=[2.1813, 2.0619, 2.0999],
up=[0.1204, -0.9852, 0.1215])
This method removes points if they don’t have a specified number of neighbors within a certain radius. It is another way to reduce noise by eliminating sparsely populated regions of the point cloud that are likely to be noise or outliers.
2.8. Convex Hull
Convex Hulls is the smallest convex polygon that can engulf the set of points in it’s space. If we have a set of pins as the points displayed below, and we would like the rubber band to snap tight arround them, the shape the rubber band makes is the convex hull.
1
2
3
4
hull, _ = chair.compute_convex_hull()
hull_ls = o3d.geometry.LineSet.create_from_triangle_mesh(hull)
hull_ls.paint_uniform_color((1, 0, 0))
o3d.visualization.draw_plotly([chair, hull_ls])
This convex hull is extremely useful in obstacle avoidance systems.
3. Point Cloud Registration Using Open3D
We can do so with registring point clouds through different approaches such as:
Iterative Closest Point (ICP): Iterative Closest Point algorithm keeps one point cloud, which is the reference, fixed, while transforming (usually a combination of translation and rotation) the other, the source, to best match the reference.
Normal distributions transform (NDT): NDT registers two point clouds by first associating a piecewise normal distribution to the first point cloud, that gives the probability of sampling a point belonging to the cloud at a given spatial coordinate, and then finding a transform that maps the second point cloud to the first by maximising the likelihood of the second point cloud on such distribution as a function of the transform parameters.
Phase Correlation: Phase Correlation is an image matching algorithm based on the Fourier shift property, which states that a translation shift between two similar images generates a linear phase difference in the Fourier frequency domain
Feature-based: In this method, stochastic or functional models try to use point, line, plane or other features as a way to estimate 3D spatial similarity transformation parameters.
In this article however, we will be looking at ICP Registration.
As mentioned above, during ICP registration, one of the two point clouds is kept as reference (source) while transforming the other (target) to roughly align the source point cloud to the target point cloud. The output is a refined transformation that tightly aligns the two point clouds.
There are two ICP variants: the point-to-point ICP, and the point-to-plane ICP.
Point-to-Point ICP vs. Point-to-Plane ICP
Point to Point ICP and Point to Plane ICP are similar in concept. The main change is in the cost function. Unlike point to point ICP, where we are looking for the closest point in the other point cloud for all of those points and then trying to minimize the square distances between them, in Point-to-Plan ICP, we additionally take the normal information into account.
We do so through computing surface normals on my target point cloud and then project the error vector and take the dot product with the euclidean distance between the points on the source and target point clouds.
As shown in the figure above, the only modification to the cost function is the addition of the dot product of the normal of target point in the PCD denoted as $n_y$, where $y_n$ is the target point in in the target PCD, and x is the source point in the source PCD.
Point Cloud Registration using Open3D
First things first, let’s import the source and target point clouds that we will be working with.
1
2
3
4
5
6
7
8
import import numpy as np
import open3d as o3d
import copy
#Read Source and Target PCD
demo_pcds = o3d.data.DemoICPPointClouds()
source = o3d.io.read_point_cloud(demo_icp_pcds.paths[0])
target = o3d.io.read_point_cloud(demo_icp_pcds.paths[1])
It’s obvious that both the source and target are point clouds of the same environment but at different angles.
Rather than using the visualisation function we are familiar with, let’s use Open3D’s helper function draw_registration_result.It visualizes the alignment during the registration process.
1
2
3
4
5
6
7
8
9
10
11
def draw_registration_result(source, target, transformation):
source_temp = copy.deepcopy(source)
target_temp = copy.deepcopy(target)
source_temp.paint_uniform_color([1, 0.706, 0])
target_temp.paint_uniform_color([0, 0.651, 0.929])
source_temp.transform(transformation)
o3d.visualization.draw_geometries([source_temp, target_temp],
zoom=0.4459,
front=[0.9288, -0.2951, -0.2242],
lookat=[1.6784, 2.0612, 1.4451],
up=[-0.3402, -0.9189, -0.1996])In this tutorial, we show two ICP variants, the point-to-point ICP and the point-to-plane ICP [Rusinkiewicz2001]_.
The function requires the source, and target point clouds as arguments. To align them on top of each other, we must also specify an inital transformation to align the point clouds on top of each.
The function above visualizes a target point cloud with yellow and a source point cloud twith cyan ransformed with an alignment transformation. The more they overlap, the better.
To get a rough image of these point cloud’s alignment, let’s specify a temperoray transformation that is obtained using a global registration algorithm.
1
2
3
4
5
trans_init = np.asarray([[0.862, 0.011, -0.507, 0.5],
[-0.139, 0.967, -0.215, 0.7],
[0.487, 0.255, 0.835, -1.4], [0.0, 0.0, 0.0, 1.0]])
draw_registration_result(source, target, trans_init)
Global and Local Registration
During the registration between the point clouds, common algorithms such as Iterative Closest Point (ICP), do not converge to the global error minimum and instead converge to a local minimum, hence ICP registration is referred to as a local registration. It relies on an approximate alignment as an initial step.
Another category of registration methods is global registration. These algorithms do not depend on alignment for initialization. Typically, they produce less precise alignment outcomes but make aid in converging to the global minimum and hence are employed as the initial step for local methods.
Evaluating Point Cloud Registration Before Applying Iterative Closest Point
With the transformation that we’ve initialised and obtained from a global registration method, we can calculate how efficient the results of global registration alone can be through the use of the function evaluate_registration. It calculates two main metrics:
- fitness: which measures the overlapping area (# of inlier correspondences / # of points in target). The higher the better.
- inlier_rmse: which measures the Root Mean Square Error of all inlier correspondences. The lower value the better the registration results are.
1
2
3
4
print("Initial alignment")
evaluation = o3d.pipelines.registration.evaluate_registration(
source, target, threshold, trans_init)
print(evaluation)
The code above calls the evaluate_registration method from the registration pipeline and takes in the source point cloud, target point cloud, threshold, and initial transformation
Applying Iterative Closest Point-to-Point
1
2
3
4
5
6
7
8
9
threshold=0.02
print("Apply point-to-point ICP")
reg_p2p = o3d.pipelines.registration.registration_icp(
source, target, threshold, trans_init,
o3d.pipelines.registration.TransformationEstimationPointToPoint())
print(reg_p2p)
print("Transformation is:")
print(reg_p2p.transformation)
draw_registration_result(source, target, reg_p2p.transformation)
Here we apply the registration_icp method of the registration pipeline. This requires the below as arguments:
- Source Point Cloud
- Target Point Cloud
- Threshold also known as max_correspondence_distance which is the maximum correspondence points-pair distance, here we specify it as 0.02.
- Initial global registration: this is optional, so if unlike this example and we don’t have a global registration transformation we can not specify it and the is: array([[1., 0., 0., 0.], [0., 1., 0., 0.], [0., 0., 1., 0.], [0., 0., 0., 1.]]). In this example, we already do have an initialisation from a global registration called “trans_init”.
- Method of registration: Either one of the two methods below:
- Registration Point to Point: registration::TransformationEstimationPointToPoint
- Registration Point to Plane:registration::TransformationEstimationPointToPlane
The default is PointToPoint without scaling. In this example we specified that we would like to use point-to-point.
Evaluating Point Cloud Registration After Applying Iterative Closest Point
The above returns “open3d.registration.RegistrationResult” which displays the fitness and RMSE score resulting from the ICP. We also displayed the result in the draw_registration_result as below:
Applying Iterative Closest Point-to-Plane
1
2
3
4
5
6
7
8
9
threshold=0.02
print("Apply point-to-point ICP")
reg_p2p = o3d.pipelines.registration.registration_icp(
source, target, threshold, trans_init,
o3d.pipelines.registration.TransformationEstimationPointToPlane())
print(reg_p2p)
print("Transformation is:")
print(reg_p2p.transformation)
draw_registration_result(source, target, reg_p2p.transformation)
Conclusion
This guide has provided a detailed overview of point cloud processing with Open3D, from basic operations like creating and visualizing point clouds to more advanced techniques such as downsampling, normal estimation, segmentation, and clustering. By applying these techniques to the PLY point cloud dataset, we’ve demonstrated how to use clustering algorithms like K-Means and DBSCAN to analyze and visualize 3D data effectively.
Open3D’s extensive features make it a powerful tool for 3D data processing. By mastering these methods, you can apply them to various applications in fields such as robotics, 3D modeling, and computer vision.