ECCV 2026
MoGe4D: Geometry-Aware Single-Image 4D Synthesis via Dense Trajectory Generation
Abstract
Generating interactive, dynamic 4D scenes from a single static image remains a core challenge. Most existing generate-then-reconstruct and reconstruct-then-generate methods decouple geometry from motion, causing spatiotemporal inconsistencies and poor generalization.
To address this, we present MoGe4D (Motion and Geometry-Aware image-to-4D synthesis), a geometry-conditioned framework that models a scene as dense 4D point trajectories. Rather than treating geometry and dynamics as two disconnected stages, MoGe4D starts from an initial geometric prior inferred from the input image and predicts future time-varying trajectories in a diffusion process, tightly coupling geometric modeling with motion generation. To support this task, we introduce TrajScene-60K, a large-scale dataset of 60,000 video samples with dense 4D point trajectories (3M+ frames, ~12B 3D point annotations). Built on this, our diffusion-based 4D Scene Trajectory Generator (4D-STraG) predicts geometry-consistent and motion-plausible trajectory fields, with a depth-guided motion normalization strategy and a Motion Perception Module (MPM). We further propose a 4D View Synthesis Module (4D-ViSM) to render the generated 4D representation into videos under arbitrary camera trajectories. Experiments show that MoGe4D produces high-quality 4D scenes with strong temporal coherence, geometry-aware consistency, and compelling novel-view synthesis from a single image.
Methodology: Geometry-Aware 4D Synthesis
MoGe4D represents a scene as dense 4D point trajectories P ∈ RT×N×3 of N = H × W points over T frames. This class-agnostic representation requires no predefined template, decoupling geometry from object category. Starting from an initial geometric prior inferred from the input image, the framework tightly couples geometric modeling with motion generation through two components:
- 4D Scene Trajectory Generator (4D-STraG): A diffusion model that predicts geometry-consistent and motion-plausible trajectory fields conditioned on the image. It adopts a Depth-Guided Motion Normalization to remove scale ambiguity by normalizing motion with the viewing frustum at the initial depth, and a Motion Perception Module (MPM) that injects motion-aware priors from the input image.
- 4D View Synthesis Module (4D-ViSM): Renders the generated 4D point-cloud representation into high-fidelity videos under arbitrary camera trajectories, coherently filling dis-occluded regions with generative priors.
TrajScene-60K Dataset
To address the scarcity of high-quality training data for scene-level 4D generation, we present TrajScene-60K, a large-scale, real-world indoor/outdoor dataset:
- 60,000 High-Quality Samples: Curated from WebVid-10M (~200K candidates) via VLM-based filtering (CogVLM2 & DeepSeek-V3) and scale-consistency validation to ensure meaningful, self-initiated motion.
- Dense Multi-Modal Annotations: Provides dense occlusion-aware 4D point trajectories (~12 billion 3D points over 3M+ frames), per-frame depth maps, and occlusion masks, obtained via dense point tracking and Gaussian Splatting rendering.
- Rich Semantics: Paired with high-quality captions describing both scene content and dynamic behavior.
Generated Samples
Input
4D Point Tracking Generated From 4D-STraG
Multi-View Videos Generated From 4D-ViSM
Prompt: A brown bear walks across rocky terrain.
Prompt: A camel walks along a path in a sunny zoo enclosure.
Prompt: A grey rhino strolls peacefully through the dappled sunlight.
Qualitative Results
Multi-View & Trajectory Generation
Our model generates consistent 4D point clouds (Top) and renders high-quality videos under arbitrary camera trajectories (Bottom).
Comparison with State-of-the-Art Single-Image-to-4D methods
Visual comparison with 4Real, DimensionX, Gen3C, and Free4D. MoGe4D produces more diverse motion and preserves structural consistency better than decoupled approaches.
Quantitative Analysis
We evaluate MoGe4D on VBench across six metrics — Subject / Background Consistency, Motion Smoothness, Dynamic Degree, Aesthetic Quality, and Imaging Quality — and further conduct a VLM-based assessment using Qwen2.5-VL-72B that scores generated 4D scenes on a 1–5 scale. Compared with optimization-, learning-, and training-free baselines (e.g., 4Real, GenXD, DimensionX), MoGe4D consistently achieves superior motion dynamics and visual quality while preserving geometry-aware consistency.
BibTeX
@inproceedings{zhang2026moge4d,
title={MoGe4D: Geometry-Aware Single-Image 4D Synthesis via Dense Trajectory Generation},
author={Zhang, Yanran and Wang, Ziyi and Zheng, Wenzhao and Zhu, Zheng and Zhou, Jie and Lu, Jiwen},
booktitle={European Conference on Computer Vision (ECCV)},
year={2026}
}