SiamesePointNet: A Siamese Point Network Architecture for Learning 3D Shape Descriptor

Abstract

We present a novel deep learning approach to extract point‐wise descriptors directly on 3D shapes by introducing Siamese Point Networks, which contain a global shape constraint module and a feature transformation operator. Such geometric descriptor can be used in a variety of shape analysis problems such as 3D shape dense correspondence, key point matching and shape‐to‐scan matching. The descriptor is produced by a hierarchical encoder–decoder architecture that is trained to map geometrically and semantically similar points close to one another in descriptor space. Benefiting from the additional shape contrastive constraint and the hierarchical local operator, the learned descriptor is highly aware of both the global context and local context. In addition, a feature transformation operation is introduced in the end of our networks to transform the point features to a compact descriptor space. The feature transformation can make the descriptors extracted by our networks unaffected by geometric differences in shapes. Finally, an N‐tuple loss is used to train all the point descriptors on a complete 3D shape simultaneously to obtain point‐wise descriptors. The proposed Siamese Point Networks are robust to many types of perturbations such as the Gaussian noise and partial scan. In addition, we demonstrate that our approach improves state‐of‐the‐art results on the BHCP benchmark.