Learning from neighborhoods for 3D point cloud classification
Date
2023
Journal Title
Journal ISSN
Volume Title
Publisher
University of Delaware
Abstract
Point clouds are a popular representation of 3D data collected from 3D or multiple-2D sensors. Non-sensor-based data, such as the 3D arrangement of atoms in enzymes, can also be represented as point clouds. Unlike 2D data that fit into grids and make application of Convolutional Neural Networks (CNNs) easier, 3D data, particularly point clouds being irregular and sparse, are challenging to apply CNNs. However, with the advent of PointNet, which directly processes raw point clouds, the popularity of deep neural networks has increased for 3D point cloud processing, particularly for classification. ☐ 3D point cloud classification is a high-level Computer Vision task with applications in autonomous driving, robotics, AR/VR, surveillance, and medical treatment. Recent deep learning models for 3D point cloud classification depend on how well they capture points from the surface patches called local neighborhoods. Standard techniques in these models, such as neighborhood querying, data augmentation, views (in view-based methods), and local features, are crucial to achieving higher classification accuracy. However, neighborhoods hold different geometric complexities, and applying the same augmentation techniques at the object level is less effective in augmenting complex local structures. Furthermore, existing view-based methods for point clouds lose accuracy at lower view resolutions due to losing finer details. Despite the availability of 3D (or multiple 2D) sensors, the availability of 3D datasets to train data-hungry deep learning models is scarce. Moreover, while learning discriminative features, existing methods need more focus on better capturing local neighborhood points. This dissertation presents comprehensive novel local neighborhood-focused techniques in classification frameworks and introduces the largest mobile phone-based 3D point cloud grocery dataset. ☐ In this dissertation, a neighborhood information-based strategy is applied to achieve higher accuracy in 3D point cloud classification. First, focusing on local neighborhood querying, I present re-oriented ellipsoid querying to capture meaningful geometrical points from the local neighborhoods. Then, I scale these ellipsoids to capture varying proportions of local neighborhoods. Next, focusing on data augmentation, I present PatchAugment, which applies different augmentations to local neighborhoods. Then, focusing on point cloud-based views, I present neighbor projections to alleviate the loss of finer details at lower resolutions of views to achieve higher accuracy or considerably regain lost accuracy. Next, I present the largest mobile phone-based point cloud grocery dataset called 3DGrocery100 and benchmark it with six recent deep-learning models for 3D point cloud classification. Then, I present four additional local features for 3D point cloud classification and show their impact on state-of-the-art methods. Finally, I again present re-oriented ellipsoid querying to better capture local neighborhoods in modeling point cloud sequences for human activity recognition.
Description
Keywords
3D classification, Deep learning, Geometric processing, Local neighborhoods, Point clouds