feat(pipeline): Support Gaussian-based surface reconstruction methods

[hummat]

Summary

Add pipeline support for surface reconstruction methods built on 3D Gaussian Splatting, bridging the gap between the current SDF-only mesh extraction path and the Gaussian splat export path (which produces splats, not meshes).

Context: Three survey papers (nerf-3dgs-survey, 3d-gs-survey, 3dgs-survey) identify Gaussian-based surface reconstruction as the dominant trend, yet mini-mesh currently has no path from Gaussians → mesh.

---

Motivation

Currently mini-mesh offers two disjoint paths:

• SDF methods (NeuS, VolSDF, Neuralangelo, etc.) → marching cubes → textured mesh ✅
• Gaussian methods (splatfacto) → splat export only, no mesh ❌

The surface reconstruction survey categorizes three families of methods that produce meshes from Gaussians, all significantly faster than SDF methods:

Family	Representative	Approach	Speed
Regularization	SuGaR, DN-Splatter	Flatten Gaussians → Poisson recon	~15 min
Opacity/Depth field	GOF, PGSR, 2DGS	Extract level set or fuse depth maps	~25 min
Stereo-based	GS2Mesh	Render stereo pairs → stereo matching → TSDF	~20 min

Compare to Neuralangelo at 5+ hours for similar quality on indoor scenes.

---

Proposed Design

Stage 1: Nerfstudio method integration

These methods are implemented as nerfstudio methods or gsplat extensions. Add training and export routing for:

Priority methods (ordered by implementation maturity and relevance):

1. 2DGS (2d-gs / splatfacto-2d) — Oriented disk primitives with intrinsic normals. Mesh via TSDF fusion of rendered depth maps. Already in gsplat as 2dgs. Regularization: depth distortion loss (α=1000) + normal consistency (β=0.05).

2. DN-Splatter — Standard splatfacto + depth/normal priors. Mesh via Poisson surface reconstruction (depth level 9) from back-projected depth/normal maps. Loss weights: λ_d=0.2, λ_n=0.1, λ_s=0.1. Available as nerfstudio plugin.

3. GOF (Gaussian Opacity Fields) — Ray-tracing-based opacity field + adaptive Marching Tetrahedra with binary search level-set. Improved densification metric (per-pixel gradient norms). ~24 min on A100. Requires CGAL + Kaolin.

4. PGSR — Flatten Gaussians into planes, unbiased depth rendering, multi-view NCC consistency. Mesh via TSDF fusion + Marching Cubes. Multi-view constraints use 8 neighboring frames at <30° angle. ~0.5–1h on 4090.

5. SuGaR — Regularization-based: minimize smallest scale axis, then Poisson recon (depth 10) on flattened Gaussian centers. Joint mesh-Gaussian refinement with barycentric binding. Available as nerfstudio plugin.

6. GS2Mesh — Post-hoc: render stereo pairs from trained splatfacto, feed to pretrained stereo model (DLNR/RAFT), TSDF fusion. Baseline = 3.5% of scene diameter. Could work with any existing splatfacto checkpoint.

Stage 2: Pipeline integration (train.sh + export.sh)

train.sh changes:

# Detect GS surface methods and route to nerfstudio
if [[ "$model_name" == *2dgs* || "$model_name" == *dn-splatter* || ... ]]; then
    # Route to ns-train with appropriate method
fi

export.sh changes:

# New branch for GS-based mesh extraction
if [[ "$model_name" == *2dgs* || "$model_name" == *dn-splatter* ]]; then
    # Method-specific mesh extraction:
    # - 2DGS/PGSR: TSDF fusion of rendered depth maps
    # - DN-Splatter/SuGaR: Poisson surface reconstruction
    # - GOF: Marching Tetrahedra
    # Then: standard texture baking (reuse sdf-texture-mesh or xatlas)
fi

Config files:

• config/2dgs-short.sh, config/dn-splatter-short.sh, etc.
• Follow existing naming convention (-test, -short, default, -long)

Stage 3: GS2Mesh as post-hoc option

GS2Mesh is unique — it works on any trained splatfacto model without retraining:

--export-method gs2mesh  # Run stereo matching on rendered pairs from existing checkpoint

This could be added as an export-time option for all *splat* models.

---

Alternatives Considered

1. SDF-only path: Keep current pipeline, rely on NeuS/Neuralangelo for all mesh extraction. Rejected: 10–100× slower training, and the field is moving toward Gaussian-based methods.

2. Custom implementations: Build surface extraction from scratch in mini-mesh. Rejected: these methods already have nerfstudio-compatible implementations; wrapping is cheaper than reimplementing.

3. Single method only: Just add 2DGS. Considered for Stage 1, but the pipeline routing is the same for all methods — once the branch exists, adding more methods is incremental config work.

---

Tasks

Stage 1: First method (2DGS or DN-Splatter)

• Verify 2DGS/DN-Splatter works with current nerfstudio fork
• Add training routing in train.sh
• Add mesh extraction routing in export.sh
• Create config files (config/2dgs*.sh or config/dn-splatter*.sh)
• Test end-to-end: video → SfM → train → mesh export
• Update docs/methods_and_models.md

Stage 2: Additional methods

• Add PGSR support (TSDF fusion path)
• Add SuGaR support (Poisson recon path)
• Add GOF support (Marching Tetrahedra path — check CGAL/Kaolin deps)
• Add method configs for each

Stage 3: GS2Mesh post-hoc export

• Add --export-method gs2mesh flag
• Implement stereo pair rendering from trained splatfacto
• Wire stereo matching model (DLNR/RAFT) + TSDF fusion
• Test on existing splatfacto checkpoints

Docker

• Ensure gsplat is available (feat(docker): Add gsplat to Docker image to avoid runtime compilation #8)
• Add any additional deps (CGAL, Kaolin for GOF; stereo models for GS2Mesh)

---

References

Survey papers

• Surface Reconstruction Survey (Xu et al., PeerJ CS) — Taxonomy of GS→mesh methods
• 3DGS Survey (Chen & Wang) — Comprehensive GS method overview
• NeRF-3DGS Evolution Survey (MDPI Sensors) — MVG→NeRF→3DGS progression

Method papers

• 2DGS — 2D oriented disk primitives, TSDF extraction
• DN-Splatter — Depth/normal priors, Poisson extraction
• GOF — Opacity field, Marching Tetrahedra
• PGSR — Planar Gaussians, multi-view consistency
• SuGaR — Flatness regularization, Poisson extraction
• GS2Mesh — Stereo matching on rendered pairs

---

Related Issues

• feat(docker): Add gsplat to Docker image to avoid runtime compilation #8 — gsplat in Docker (prerequisite for all GS methods)
• feat(export): PBR-ish texture extraction and BRDF-aware training #11 — PBR texture extraction (benefits from clean GS-based meshes)
• feat(export): Add optional mesh retopology step #6 — Retopology (applicable to GS-extracted meshes too)
• feat(pipeline): End-to-end validation with ground truth meshes #5 — E2E validation (need GS surface metrics)