GPLD3D: Latent Diffusion of 3D Shape Generative Models by Enforcing Geometric and Physical Priorse

CVPR2024
1Institute for Intelligent Computing, Alibaba Group, 2The University of Texas at Austin Austin 

(Left) Synthetic shapes generated by 3DS2VS, which present various issues in geometric feasibility and physical stability. (Right) Synthetic shapes generated by GPLD3D, which have significantly improved geometric feasibility and physical stability.

Abstract

State-of-the-art man-made shape generative models usually adopt established generative models under a suitable implicit shape representation. A common theme is to perform distribution alignment, which does not explicitly model important shape priors. As a result, many synthetic shapes are not connected. Other synthetic shapes present problems of physical stability and geometric feasibility. This paper introduces a novel latent diffusion shape-generative model regularized by a quality checker that outputs a score of a latent code. The scoring function employs a learned function that provides a geometric feasibility score and a deterministic procedure to quantify a physical stability score. The key to our approach is a new diffusion procedure that combines the discrete empirical data distribution and a continuous distribution induced by the quality checker. We introduce a principled approach to determine the tradeoff parameters for learning the denoising network at different noise levels. Experimental results show that our approach outperforms state-of-the-art shape generations quantitatively and qualitatively on ShapeNet-v2.

Methodology

MY ALT TEXT

Overview of GPLD3D. It employs a quality checker that assesses the geometric feasibility and physical stability scores of synthetic shapes. This quality checker guides the diffusion procedure to sample in regions of the latent space that correspond to shapes that pass the quality checker and avoid regions that do not pass the quality checker. The backbone is 3DShape2VecSet.

Video

Gallery Results

BibTeX

@InProceedings{Dong_2024_CVPR,
    author = {Dong, Yuan and Zuo, Qi and Gu, Xiaodong and Yuan, Weihao and Zhao, Zhengyi and Dong, Zilong and Bo, Liefeng and Huang, Qixing},
    title = {GPLD3D: Latent Diffusion of 3D Shape Generative Models by Enforcing Geometric and Physical Priors},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month = {June}, year = {2024}, pages = {56-66} }
}