Skip to content

Conversation

@billybjork
Copy link
Owner

Summary

Introduces a new /test-2 route that explores an alternative rendering approach for RGBD sprite sheet animations. Instead of the mesh-based displacement technique used in /test, this implementation treats RGBD data as a point cloud, where each pixel becomes an independent 3D point. This eliminates mesh stretching at depth discontinuities while creating a visually compelling aesthetic where depth edges naturally dissolve into particle scatter.

Key Changes

  • New route & template: routers/test_2.py and templates/test-2.html following existing patterns
  • Point cloud renderer: Replaces DepthThumbnail mesh geometry with THREE.BufferGeometry + THREE.Points, using vertex/fragment shaders to sample RGBD atlases and position points in 3D space
  • Enhanced camera control: Adds mouse parallax tracking (X/Y offset) combined with scroll-based tilt, with smooth lerp interpolation for immersive perspective shifts
  • Mobile support: Optional device orientation (gyroscope) integration for mobile parallax
  • Configurable point cloud styling:
    • Point density (stride-based sampling), size, shape (square/circle/soft), and attenuation
    • Depth-based sizing and opacity for visual depth cueing
    • Edge scatter effect at depth discontinuities using noise-based dispersion
    • Depth of field simulation and multiple color modes (RGB, depth-colorized, normal-colorized)
  • Reuses existing systems: Animation loop, scroll-based frame advancement, asset loading, and control panel UI patterns from /test
  • Updated main.py: Registers the new router

Implementation Details

  • Vertex shader approach: UV grid buffer (static) + frame index uniform allows efficient frame changes without buffer updates; shaders sample both RGB and depth atlases to compute 3D positions and colors
  • Performance: ~115K points per thumbnail at 0.5 density (640x360), well within Three.js budget for 60fps
  • Visual direction: Creates a "living photograph dissolving into particles" aesthetic — smooth regions appear continuous while depth edges naturally scatter, reinforcing volumetric perception
  • Mouse tracking: Normalized cursor position (-1 to 1) drives camera X/Y offset with configurable strength; smoothly returns to center on mouse leave
  • No asset pipeline changes: Reuses all existing sprite sheets and metadata from static/test/rgbd-sprites/

https://claude.ai/code/session_01ESEahvkYKaUe1SzmYVebqc

Detailed prompt for creating a point cloud renderer that reuses existing
RGBD sprite sheet assets but renders them as THREE.Points instead of
displaced meshes, with mouse parallax, edge scatter effects, and
depth-adaptive point sizing.

https://claude.ai/code/session_01ESEahvkYKaUe1SzmYVebqc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants