Working with local timing source

[balintlaczko]

Hi!

First of all, thanks for this great-great project! I am able to synchronize lots of different videos with MCorp.mediaSync and a "cloud-based" shared motion, with acceptable deviation in sync. The project is about synchronizing video scores for musicians (where the video scores contain audio click-tracks with all kinds of cues as well), so when I say "acceptable" it is in the sense of musical synchrony, for less time-sensitive purposes it's perfect.

For the final performance, I need to synchronize those videos with an audio file played back from the host computer (in Max). So the position of that audio file should be the "motion" for all the clients. Here is the way I am doing it:

• creating a local websocket server in Node (in Max)
• connecting to that with all my clients
• each client initializes a local TimingObject to itself
• I am updating all those client-side TimingObject-s with vectors (f.ex. {position: 12.345, velocity: 1}) from the Node server. The server is embedded in a Max patch, and gets the position values from the playback of the audio file.

This so far seems to work OK, though I still have to test if it adds some latency or any other kind of instability to the mix. What I am a bit concerned about is that the API docs say that the .update() method should not be used too frequently, and I am doing it on a rate of 10Hz or higher... The side effect of this is a constant heavy artifact in some browsers (typically on mobile, desktops seem to be fine) which implement variable playback rate. (If I set the MCorp.mediaSync to mode: 'skip' I get almost constant skipping...)

My hunch tells me that this is not the way I should be doing this (it kinda works, but I don't want those nasty audio artifacts..). Do you have any suggestions how I could improve or change this method?

Thanks a lot!
Balint

[ingararntzen]

Hi - and thanks!

The way I read this you want to switch from online sync server to local sync server for better performance. If so, I doubt that is going to be worth the trouble. The dominant source of sync issues isn't the network part of things, it is the video playback (mediasync). In practice it doesn't really matter where the sync server is.

I think you are right to be concerned about the approach. The point of timing objects is to provide a predictable motion -- that's what velocity is for, and then to do gentle changes, not too often. If you are just sending frequent position updates - you are working against that goal, constantly interrupting the media elements, which they absolutely don't like.

If you want to use your audio playback as a source for motion in your system, you should not update the timing object all the time. Instead you need to implement functionality for continuously monitoring the difference between the audio playback clock and your timing object. When the difference grows beyond a certain limit, you need to take action to realign the timing object with your audio playback. Importantly though, realignment should be performed by adjusting the velocity, not the position, aiming to reduce the difference gradually over time - as opposed to instantly. If your algorithm is good, it will converge on a timing object velocity which is not exactly 1.0, but fairly close - matching the effective playback rate of your audio system.

Hope this helps :)

Ingar

[balintlaczko]

OK! Thanks a lot. I thought this mechanism is built into the mediaSync, but now I understand that it is probably the online motion which measures and reports back these differences and adjusts the client when you are using one? So in case my "motion" is on LAN it is my responsibility to implement the difference adjustment algo?

[ingararntzen]

If you are on a LAN, you'll need your own server on the LAN for keeping client-side timing objects in sync. That is a separate problem though. The issue I talked about was limited to the client where you do the audio playback, i.e. making sure a local timing object is aligned to your local audio clock.

[balintlaczko]

Yes, that is what I am doing now. I slowly start to understand that I basically had to implement a similar adjustment algorithm between my audio playback (the "master" motion) and the client-side timingObjects as what's already implemented between the video tags and those (client-side) timingObjects. It seems to work much better now, it almost feels slightly better than the cloud-based version - though I need some testing to verify this. On desktop and android browsers it feels very nice, but on ios I get a lot of skipping. That might get fixed if I loose up the sync, but it definitely needs some tinkering...

Tusen takk!! :))