feat(export): PBR-ish texture extraction and BRDF-aware training

[hummat]

Summary

Add support for extracting PBR-compatible textures (basecolor, roughness, normal, metallic) from trained models, with optional BRDF-aware training signals.

Learning context: hummat/learning-blender#11
Local docs: docs/pbr_learning_starter_pack.md, docs/brdf_and_shading_effects.md

---

Staged Approach

Stage 1: Proxy PBR from Ref-NeRF-style Signals ✅ (partially done)

SDFStudio already supports Ref-NeRF-inspired flags:

• use_diffuse_color — diffuse/specular decomposition
• use_specular_tint — colored specular (metallic-ish)
• use_reflections — reflection-direction encoding
• use_n_dot_v — Fresnel-like angular cues
• enable_pred_roughness — learned roughness parameter

Current state: These produce "proxy" PBR signals — better structured than raw NeRF radiance, but NOT physically calibrated GGX roughness.

Tasks:

• Verify enable_pred_roughness propagates to texture export
• Document which config enables all Ref-NeRF signals
• Add config preset for "PBR-ish" training (e.g., --data.pipeline neus-facto-pbr)

Stage 2: glTF/GLB Export with PBR Textures

Export trained models to standard PBR format usable in Blender, game engines, web viewers.

Output format (glTF 2.0 metal-rough):

Texture	Channel	Encoding
baseColorTexture	RGB	sRGB
normalTexture	RGB	Linear, OpenGL (+Y)
metallicRoughnessTexture	R=AO, G=Roughness, B=Metallic	Linear

Tasks:

• Implement GLB exporter in scripts/export.sh or Python
• Wire basecolor from diffuse texture (apply sRGB, clamp)
• Wire roughness from enable_pred_roughness output
• Default metallic = 0 (constant); allow override
• Pack ORM texture (AO optional, roughness required, metallic=0)
• Ensure normal map uses correct convention (flip green if needed)
• Compute MikkTSpace tangents on export
• Add --export-format glb flag to run.sh
• Smoke test: import into Blender, check under HDRI

Export variants (for downstream refinement):

• basecolor_srgb.png — display-ready
• basecolor_linear.exr — unclamped linear for optimization init

Stage 2.5: UV-Space Inverse Rendering Refinement (Optional)

For "true-ish" PBR from real video under unknown lighting. Uses Stage 1 exports as initialization.

Pipeline:

frames + poses + mesh/UVs → UV observation builder → GGX fitting → refined PBR maps

Tasks:

• Implement scripts/pbr_uv_observations.py — collect per-texel multi-view samples
• Implement Stage 2.5a: Lambert + SH + exposure baseline fit
• Implement Stage 2.5b: GGX dielectric (fixed F₀≈0.04, metallic=0)
• Add TV/smoothness regularization in UV space
• Use Stage 1 exports as initialization (prefer linear basecolor)
• Output refined PBR maps

Dependencies:

• PyTorch differentiable GGX forward model
• Per-view lighting estimation (SH or small env basis)

Stage 3: External Tool Integration (Future)

For highest quality or specific use cases.

Tool	Use Case	Notes
nvdiffrec	Material-only refinement on frozen geometry	Requires #7
Material Palette	Per-view SVBRDF prediction + UV aggregation	Generative prior
DualMat	Single-image PBR with diffusion prior	Newer, dual-path

Tasks:

• Add optional nvdiffrec refinement step (after feat(docker): Add nvdiffrast to Docker image for faster texturing #7)
• Investigate Material Palette / DualMat integration

---

Technical Decisions

Color Space

• Basecolor: Export both sRGB (display) and linear (optimization)
• Roughness/Metallic/Normal: Always linear
• Warning: Current linear_to_srgb().clamp() in diffuse export loses information for refinement

Roughness Semantics

• Stage 1 roughness is a proxy — learned to help view-dependent appearance, not calibrated to GGX
• Stage 2.5 roughness is physically grounded (GGX microfacet)
• Document this distinction clearly

Metallic Handling

• Default to constant 0 (dielectric assumption)
• Most real-world objects are dielectric; metallic textures rarely needed
• Allow --metallic 1 override for metal objects

Normal Map Convention

• Default: OpenGL (+Y up)
• Add --normal-flip-y for DirectX convention
• Ensure tangents are MikkTSpace

---

Alternatives Considered

1. External tools only: Skip internal PBR support entirely, rely on nvdiffrec/Material Palette post-processing. Rejected because: adds mandatory external dependency, loses integration benefits, and proxy PBR from existing Ref-NeRF flags is nearly free.

2. Diffuse-only export: Keep current simple texture export without PBR channels. Rejected because: limits downstream usability in game engines/renderers that expect PBR materials.

3. Pre-trained material networks: Use off-the-shelf SVBRDF estimation (e.g., MaterialGAN) directly on rendered views. Considered as Stage 3 option (Material Palette, DualMat) but not primary path because: requires additional models, less integrated with reconstruction pipeline.

---

Validation

• Export GLB, import into Blender, render under HDRI — check plausibility
• Compare roughness behavior: shiny vs matte regions should match source
• Relight test: exported materials should respond correctly to new lighting

---

References

• docs/pbr_learning_starter_pack.md — learning resources
• docs/brdf_and_shading_effects.md — concept cheat sheet
• docs/missing_brdf_papers_analysis.md — inverse rendering papers
• LearnOpenGL PBR — reference GGX implementation
• Brian Karis — Real Shading in UE4 — metal-rough model

---

Related Issues

• feat(docker): Add nvdiffrast to Docker image for faster texturing #7 — nvdiffrec for Docker (enables Stage 3)
• feat(export): Add optional mesh retopology step #6 — Retopology (clean mesh helps texturing)