Sync4DGS: Dynamic 3D Scene Reconstruction from Any Unsynchronized Multi-View Videos

Semyeong Yu · Munchurl Kim

A novel view synthesis result of Sync4DGS.

Summary: Sync4DGS with 3D trajectory-driven time alignment

Contents

1. Setup
2. Preprocess Datasets
3. Training
4. Evaluation

Setup

Environment Setup

Clone the source code of this repo.

mkdir sync4dgs
cd sync4dgs
git clone --recursive https://github.com/KAIST-VICLab/Sync4DGS.git .

Installation through pip is recommended. First, set up your Python environment:

conda create -n sync4dgs python=3.9
conda activate sync4dgs

Make sure to install CUDA and PyTorch versions that match your CUDA environment. We've tested on RTX 4090 GPU with PyTorch version 2.1.2. Please refer https://pytorch.org/ for further information.

pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

Modify prefix of environment.yaml to your conda environment path. Then the remaining packages can be installed with:

pip install --upgrade setuptools cython wheel
pip install -r requirements.txt
conda env update --file environment.yml

Preprocess Datasets

For dataset preprocessing, we follow STG.

Neural 3D Video Dataset

First, download the dataset from here. You will need colmap environment for preprocess. To setup dataset preprocessing environment, run scrips:

./scripts/env_setup.sh

To preprocess dataset, run script:

./scripts/preprocess_all_n3v.sh <path to dataset>

Technicolor dataset

Download the dataset from here. To setup dataset preprocessing environment, run scrips:

./scripts/preprocess_all_techni.sh <path to dataset>

Please refer STG for further information.

Training

Run command:

python train.py --config configs/<some config name>.json --model_path <some output folder>  --source_path <path to dataset>

Evaluation

Run command:

python render.py --model_path <path to trained model>  --source_path <path to dataset> --skip_train --iteration <trained iter>