UniPre3D: Unified Pre-training of 3D Point Cloud Models with Cross-Modal Gaussian Splatting

Ziyi Wang*, Yanran Zhang*, Jie Zhou Jiwen Lu†
Department of Automation, Tsinghua University, China
CVPR 2025
*Indicates Equal Contribution
†Indicates Corresponding Author
Paper Supplementary Code arXiv

Abstract

The scale diversity of point cloud data presents significant challenges in developing unified representation learning techniques for 3D vision. Currently, there are few unified 3D models, and no existing pre-training method is equally effective for both object- and scene-level point clouds. In this paper, we introduce UniPre3D, the first unified pre-training method that can be seamlessly applied to point clouds of any scale and 3D models of any architecture. Our approach predicts Gaussian primitives as the pre-training task and employs differentiable Gaussian splatting to render images, enabling precise pixel-level supervision and end-to-end optimization. To further regulate the complexity of the pre-training task and direct the model's focus toward geometric structures, we integrate 2D features from pre-trained image models to incorporate well-established texture knowledge. We validate the universal effectiveness of our proposed method through extensive experiments across a variety of object- and scene-level tasks, using diverse point cloud models as backbones.

Introduction

MY ALT TEXT
Existing object-level pre-training methods usually follow a generative masked auto-encoding (MAE) paradigm. Their scene-level counterparts mostly leverage the contrastive learning paradigm. We propose a unified pre-training method that is applicable and effective to both object- and scene-level point clouds and models.

Our proposed pre-training task involves predicting Gaussian parameters from the input point cloud. The 3D backbone network is expected to extract representative features, and 3D Gaussian splatting is implemented to render images for direct supervision. To incorporate additional texture information and adjust task complexity, we introduce a pre-trained image model and propose a scale-adaptive fusion block to accommodate varying data scales.

MY ALT TEXT
UniPre3D pre-training pipeline.

Visualization

Below is visualization of UniPre3D pre-training outputs. The first row presents the input point clouds, followed by the reference view images in the second row. The third row displays the rendered images, which are supervised by the ground truth images shown in the fourth row. In the rightmost column, we illustrate a schematic diagram of the view selection principle for both object- and scene-level samples.

MY ALT TEXT
Visualization of UniPre3D pre-training outputs.

Experiments

MY ALT TEXT
Partial experimental results under object-level.
MY ALT TEXT
Partial experimental results under scene-level

More Visualization

MY ALT TEXT
More visualization of UniPre3D object-level outputs.
MY ALT TEXT
More visualization of UniPre3D scene-level outputs.

BibTeX

@inproceedings{wang2025unipre3d,
        title={UniPre3D: Unified Pre-training of 3D Point Cloud Models with Cross-Modal Gaussian Splatting},
        author={Wang, Ziyi and Zhang, Yanran and Zhou, Jie and Lu, Jiwen},
        booktitle={Proceedings of the Computer Vision and Pattern Recognition Conference},
        pages={1319--1329},
        year={2025}
      }