Skip to content

Add real-time pipeline, biofeedback fusion, embodied interaction, and evaluation modules#10

Merged
curiousbrutus merged 3 commits intomainfrom
copilot/analyze-repository-capabilities
Feb 14, 2026
Merged

Add real-time pipeline, biofeedback fusion, embodied interaction, and evaluation modules#10
curiousbrutus merged 3 commits intomainfrom
copilot/analyze-repository-capabilities

Conversation

Copy link
Contributor

Copilot AI commented Feb 14, 2026

The repository needed to demonstrate capability for WP1 MishMash doctoral research in real-time human-AI co-creation for musical performance. The existing system had strong foundations (hybrid agent, DDSP synths, EEG mapper) but lacked the streaming architecture, multimodal biofeedback integration, embodied interaction, and research evaluation infrastructure that WP1 specifically requires.

New modules

  • performance_system/realtime/ — Event-driven streaming pipeline: EventBus (thread-safe pub/sub with latency tracking), StreamingPipeline (configurable stage chain with per-stage metrics), PerformanceState (session state with human-AI synchrony computation)

  • performance_system/biofeedback/ — Multimodal physiological signal fusion: MultimodalFusion (weighted fusion with staleness detection and confidence decay), HRVProcessor (SDNN/RMSSD from IBI), GSRProcessor (tonic/phasic decomposition), PhysiologicalMapper (direct, affective circumplex, adaptive strategies)

  • performance_system/embodied/ — Gesture and spatial interaction: GestureController (classifies sustained/percussive/sweeping/tremolo from 3D position), SpatialAudioMapper (position→pan/reverb/brightness), InteractionModeManager (responsive/collaborative/autonomous/mirroring with smooth blending)

  • performance_system/evaluation/ — Research metrics: InteractionMetrics (synchrony via Pearson correlation, latency percentiles, adaptation rate, coherence), SessionLogger (timestamped frames with JSON export)

Usage

from performance_system.realtime import EventBus, StreamingPipeline, PipelineStage
from performance_system.biofeedback import MultimodalFusion, ModalityConfig, PhysiologicalMapper
from performance_system.embodied import InteractionModeManager, InteractionMode
from performance_system.evaluation import InteractionMetrics, SessionLogger

fusion = MultimodalFusion()
fusion.add_modality(ModalityConfig(name='eeg', weight=1.0, n_features=2))
fusion.add_modality(ModalityConfig(name='hrv', weight=0.8, n_features=1))

fusion.update('eeg', {'arousal': 0.7, 'focus': 0.5})
fusion.update('hrv', {'relaxation': 0.6})
state = fusion.get_fused_state()  # weighted, staleness-aware fusion

Tests

59 new tests across 4 files (70 total). All passing. Version bumped to 0.2.0.


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits February 14, 2026 18:30
… evaluation modules with 59 tests

Co-authored-by: curiousbrutus <115472478+curiousbrutus@users.noreply.github.com>
Co-authored-by: curiousbrutus <115472478+curiousbrutus@users.noreply.github.com>
Copilot AI changed the title [WIP] Analyze repository for doctoral research in interactive AI Add real-time pipeline, biofeedback fusion, embodied interaction, and evaluation modules Feb 14, 2026
Copilot AI requested a review from curiousbrutus February 14, 2026 18:36
@curiousbrutus curiousbrutus marked this pull request as ready for review February 14, 2026 19:07
@curiousbrutus curiousbrutus merged commit 9fd580f into main Feb 14, 2026
1 check failed
Copilot stopped work on behalf of curiousbrutus due to an error February 14, 2026 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants