Skip to content

Add WebRTC preview pipeline support for remote UI monitoring #11

@andrescera

Description

@andrescera

Summary

Add a parallel preview path to the GStreamer pipeline that streams low-latency video and audio to a remote frontend (CeraUI) via WebRTC, enabling real-time device preview and audio level monitoring without disrupting the main SRT encoding path.

Background

Currently, ceracoder pipelines have a single output leg: appsink → SRT. The frontend runs remotely from the capture hardware, so users cannot preview what's being captured or monitor audio levels in real-time.

Proposed Solution

  1. Pipeline branching with tee: Add a tee element before the main encoder to branch preview media
  2. Preview encoder leg:
    • Video: Scale to 640x360 @ 15-24fps, encode with lightweight codec (VP8/H264 baseline, ~300-800 kbps)
    • Audio: Include level element (message interval ~100ms) for RMS/peak data + Opus encoding
  3. WebRTC integration: Route preview through webrtcbin (sendonly)
  4. Signaling: Expose SDP/ICE over existing backend signaling channel
  5. Device selection lifecycle: Start/stop preview branch when devices change

Example Pipeline Sketch

v4l2src ! ... ! tee name=t 
  t. ! queue ! <main encode path> ! appsink name=appsink
  t. ! queue ! videoconvert ! videoscale ! video/x-raw,width=640,height=360,framerate=15/1 ! vp8enc target-bitrate=500000 ! webrtcbin name=preview_video

alsasrc ! tee name=at
  at. ! queue ! <main audio path>
  at. ! queue ! level message=true interval=100000000 ! opusenc ! webrtcbin name=preview_audio

Acceptance Criteria

  • Preview pipeline variant can coexist with existing SRT pipelines
  • Preview leg uses separate, lightweight encoder (doesn't steal main encoder resources)
  • WebRTC signaling contract defined (SDP/ICE format)
  • Audio level messages forwarded to signaling channel
  • Pipeline validation accepts preview-enabled variants
  • TypeScript bindings extended for preview pipeline generation

Related

  • CeraUI issue for frontend preview UI components

Notes

  • Keep preview bitrate budget low (300-800 kbps) to avoid impacting main SRT stream
  • Allow configurable preview resolution (default 640x360, optional 720p)
  • Consider making preview optional/togglable at runtime

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions