CIF | Consistent Instance Field for Dynamic Scene Understanding

CIF addresses key challenges in 4D instance segmentation: inconsistent instance supervision, opacity vs occupancy misconception, and semantic sparsity

Consistent Instance Field (CIF) addresses key challenges in 4D Gaussian Splatting for instance understanding: (1) weak to inconsistent instance supervision, (2) opacity vs. occupancy misconception, and (3) semantic sparsity in meaningful regions — achieving cleaner, more consistent instance segmentation across dynamic scenes.

Abstract

Understanding dynamic 3D scenes at the instance level is crucial for applications ranging from autonomous navigation to augmented reality. While 4D Gaussian Splatting has emerged as a powerful representation for novel view synthesis of dynamic scenes, lifting it to instance-aware understanding remains challenging due to inconsistent instance supervision across views, confusion between opacity and occupancy during rendering, and sparse semantic coverage in regions with limited Gaussian density.

We introduce the Consistent Instance Field (CIF), a unified representation that addresses these challenges through three key innovations: Field-Aware Splatting that properly handles identity distribution during rendering, Instance-Guided Resampling that redistributes Gaussians to ensure adequate coverage of semantic regions, and Instance Identity Estimation with calibrated probability distributions for robust instance assignment.

Our approach enables consistent panoptic segmentation and open-vocabulary 4D querying across dynamic scenes, achieving state-of-the-art results on challenging benchmarks including HyperNeRF and Neu3D datasets.

Key Contributions

🎯

Field-Aware Splatting

Novel rendering approach that properly handles identity probability distributions along pixel rays, avoiding the opacity-occupancy confusion in traditional splatting.

📊

Calibrated Identity Estimation

Robust instance assignment through calibrated probability distributions, handling inconsistent supervision across different viewpoints and timesteps.

🔄

Instance-Guided Resampling

Adaptive redistribution of Gaussian primitives based on instance importance, moving density from redundant regions to semantically meaningful areas.

🔍

Open-Vocabulary Querying

Support for language-guided 4D scene understanding, enabling natural language queries for instance selection and tracking in dynamic scenes.

Method Overview

CIF Pipeline: Instance Field combined with Radiance Field, processed through Field-Aware Splatting and Instance-Guided Resampling

CIF Pipeline. We augment 4D Gaussian Splatting with an Instance Field alongside the Radiance Field. Field-Aware Splatting renders identity distributions with proper probability handling. Instance Identity Estimation uses calibration to handle cross-view inconsistencies. Instance-Guided Resampling redistributes Gaussians from dense redundant regions to sparse but semantically important areas.

Results

Panoptic Segmentation

Instance-level segmentation on dynamic scenes

HyperNeRF

Panoptic Segmentation

Consistent instance tracking across views

Neu3D

Open-Vocabulary 4D Querying

Language-guided instance selection in 4D

HyperNeRF

Citation

@article{wu2025cif, title={Consistent Instance Field for Dynamic Scene Understanding}, author={Wu, Junyi and Nguyen, Van Nguyen and Planche, Benjamin and Tao, Jiachen and Sun, Changchang and Gao, Zhongpai and Zhao, Zhenghao and Choudhuri, Anwesa and Zhang, Gengyu and Zheng, Meng and Wang, Feiran and Chen, Terrence and Yan, Yan and Wu, Ziyan}, journal={arXiv preprint arXiv:2512.14126}, year={2025} }