Audio streaming support

[Zero3K]

It would be nice if it could also stream the audio being played by the PC that is running it as a server.

[lammel-hub]

Perhaps it is not sufficient, but a potential workaround for your needs @Zero3K, would be the Dayon built-in link to Jitsi Meet (click on the fingerprint during a session) to get sound between the two machines.

Not the same, but hopefully it can provide some value to your use case.

[lammel-hub]

@RetGal longer-shot idea: perhsps is it time to consider implementing webRTC as the protocol for Dayon! itself in the future, unless adding sound inside the current protocol is a lot easier, if course?

I believe WebRTC could be suitable, with the following caveats:

• no idea if it is much heavier on bandwidth requirements than current protocol, but probably is.
• do not know if current user adjustments could be kept intact, e.g. b/w instead of color to reduce bandwidth use
• positive side: could include the Jitsi functionality natively instead of relying on third party app
• perhaps it is very complex to implement?

Overview of WebRTC

WebRTC is primarily a peer-to-peer (P2P) technology, enabling direct data exchange between devices for audio, video, and other data streams. However, servers are often involved in the process for specific functions:

1. Signaling Servers: These facilitate the initial exchange of connection details (e.g., SDP and ICE candidates) to establish the P2P connection but are not involved in the actual data stream.

2. NAT Traversal Servers: STUN or TURN servers help overcome network obstacles like firewalls or NATs. TURN servers may relay data if direct P2P communication fails.

3. Media Servers (Optional): For group calls, recording, or broadcasting, media servers may be used to process and relay streams.

Once the connection is established, media and data typically flow directly between peers, which means I believe Dayon! perhaps could keep relying on only RVS (evidently called "signalling server" in WebRTC lingo).

When you have found a solution for the CGNAT issue, there seems not to be a need for "NAT traversal servers" to relay data either, so that seems really good to keep server cost down.

Useful for this type of app?

WebRTC can be used to remotely control a mouse, send keyboard commands, and share clipboard data. This is feasible using WebRTC's RTCDataChannel API, which allows bidirectional transfer of arbitrary data between peers. Here’s how each functionality can be implemented:

1. Mouse Control:
   Mouse events (e.g., position, clicks) can be captured on one device and sent to the remote peer via the RTCDataChannel. The receiving application can then simulate these events on the remote machine.

2. Keyboard Commands:
   Keypress events can similarly be captured and transmitted through the data channel. The remote system can interpret these commands to simulate keyboard input.

3. Clipboard Sharing:
   Clipboard content (text or files) can be shared by sending it over the RTCDataChannel. Extensions like "Sharing Clipboard" demonstrate this capability using WebRTC for secure clipboard synchronization.

Open source Java APIs

1. WebRTC-Java

   • A Java wrapper for the WebRTC Native API, enabling RTC application development for desktop platforms using Java.
   • Dependency for Maven and Gradle is available for easy integration[1].

2. Ripple-WebRTC-Server

   • A Java-based WebRTC media server built with the Helidon SE framework. It provides client SDKs and supports efficient WebRTC communication[3].

3. Ant Media Server

   • Supports WebRTC peer-to-peer communication along with 1-N and N-N communication. It provides SDKs for mobile and web platforms to facilitate P2P connections[4].

Citations:
[1] WebRTC for desktop platforms running Java - GitHub https://github.com/devopvoid/webrtc-java
[2] PeerJS - Simple peer-to-peer with WebRTC https://peerjs.com
[3] kinsleykajiva/Ripple-WebRTC-Server: This is a Java based ... - GitHub https://github.com/kinsleykajiva/Ripple-WebRTC-Server
[4] How to Create WebRTC Peer To Peer Communication - GitHub https://github.com/ant-media/Ant-Media-Server/wiki/How-to-Create-WebRTC-Peer-To-Peer-Communication
[5] WebRTC API - MDN Web Docs - Mozilla https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API
[6] Getting started with peer connections - WebRTC https://webrtc.org/getting-started/peer-connections
[7] WebRTC Native Code https://webrtc.github.io/webrtc-org/native-code/