Implement a GUI for video sync mode

[subraizada3]

Video sync mode is fully implemented in the library. We need a program that captures the screen, converts it to colors for the 48-LED ring, and uses the library to send that data out to the monitor.

[Maxr1998]

I wrote a software a few years ago that did a similar thing, just with an Arduino and an addressable RGB LED strip. I don't consider the code particularly good, but parts of the color extraction code could possibly be re-used or at least serve as a reference. I did maybe exaggerate the parallelization a bit…

[imkebe]

I made an early implementation using Prismatic

1. You need to install from https://github.com/psieg/Lightpack
2. Setup VirtualLed using settings:
   

• Set the number of LEDs - Must be 48
• Split by top/side/bottom - I recommend 16/8/16
• Change start offset to 24
• Set thickness about 10% - to avoid overflow of each grabbing area and overall amout of pixels for performance reasons
• Set skip corners since we have a ring shaped LEDs
• Side margin is to be set when having ultrawide like 38gn950 to grab from screen center. If you own 27gn you can leave it or set smaller value.
• click APPLY
• Continue x2 until it finish.

4. Rest of my Prismatic settings:
   I obtain best responsiveness vs cpuload usnig 10ms interval. Less gives noting in terms of real sync delay but peaks load. More gives noticable lag.

You need to enable HTTP API.
Right now apikey is not implemented so leave it blank.

Tick to update only when changes - it reduces cpu load.

Last thing! 27gn950controller (if using Lightsync only) GUI will fail if :

• Prismatic is turned off
• API is not enabled
• Fiewall/OS blocks 3636 TCP port

Avoiding that yet need to be implemented.

[imkebe]

To start sync run python3 ./gui.py and click Lightsync (Prismatic)

GUI/cmd need to be implemented as a daemon but it's future...

[koshisan]

Hi!
I wanted to test with my multi monitor setup here, but I seem to be missing a dependency:

Traceback (most recent call last):
File "guisync.py", line 18, in
from lightsync import LightsyncPrismatic
ModuleNotFoundError: No module named 'lightsync'

(It's either that, or I made a mistake hacking the files together from the pull request...)

[imkebe]

First PR lacks that file. Second one should be ok.

[imkebe]

I also tested Hyperion and it's Websocket API. Overall nice experience in terms of tool. API works, provide stream of RGB values for all leds. However updates for color changes are limited to 150ms. So basically it works but with unacceptable lag, wich is frustrating. If somebody plans to integrate lib with Hyperion it's need to be done via providing this library wrapper as a virtual device. Second solution is to implement here kind of virtual client for one of already provided devices (ex. UDPraw or file). File might be interesting but i can setup there only unix style filepath like default /dev/null so I don't what to expect for Windows

from re import L
import hid
import websocket
import json
import _thread
import time

from lib27gn950 import *

monitors = find_monitors()

def rgb_to_hex_safe(rgb):
    r, g, b = rgb
    if r == 0 : r = 1
    if g == 0 : g = 1
    if b == 0 : b = 1
    return '%02x%02x%02x' % (r,g,b)

def on_message(ws, message):
    #inspect(message)
    js = json.loads(message)
    ledarray = js['result']['leds'] or None
    if ledarray:
        colors = []
        for i in range(0,48*3,3):
            y = i+3
            rgb = ledarray[i:y]
            colors.insert(int(i/3),rgb_to_hex_safe((rgb[0],rgb[1],rgb[2])))
    
    with hid.Device(path=monitors[0]['path']) as dev:
        send_video_sync_data(colors, dev)
                

def on_error(ws, error):
    print(error)

def on_close(ws, close_status_code, close_msg):
    print("### closed ###")

getcolorscmd = {
  "command":"ledcolors",
  "subcommand":"ledstream-start"
}

def on_open(ws):
    def run(*args):
        with hid.Device(path=monitors[0]['path']) as dev:
            send_command(control_commands['color_video_sync'], dev)
        ws.send(json.dumps(getcolorscmd))
        time.sleep(120) # two minutes
        ws.close()
        print("thread terminating...")
    _thread.start_new_thread(run, ())

#websocket.enableTrace(True)
ws = websocket.WebSocketApp("ws://localhost:8090/",
                            on_open=on_open,
                            on_message=on_message,
                            on_error=on_error,
                            on_close=on_close)
ws.run_forever()

[koshisan]

Just a note: This added a dependency on rich, which would need to be included in the documentation/install instructions.

It now does run but, reports "No monitors found" - I am running two 27GN950 at the moment. Is this expected behaivour?

I am not familiar with python, but find_monitors looks like it should work with multiple matches...

[imkebe]

Just a note: This added a dependency on rich, which would need to be included in the documentation/install instructions.

It now does run but, reports "No monitors found" - I am running two 27GN950 at the moment. Is this expected behaivour?

I am not familiar with python, but find_monitors looks like it should work with multiple matches...

Don't consider this as a production release. Just testing and sharing with others. I do not have multiple LG monitors.

Until we don't choose how to implement lightsync - multimonitors can work only for other modes. With multiple screens you need to map leds for each one. it depends how the external tool handle multiple screens - is it just one virtual 96 led stream for two or do we need to made connection to two stream instances each one with their own 48 stream.

So right now best scenario in my opinion is to do some testing with multiple screen grabbing solutions and decide how to write general interface for this library or just take different approach when there are some constraints.

[imkebe]

Another approach for Hyperion - using raw UDP socket as a device (UDPraw). It's better than via websocket however screen grabbing is slower and more CPU consuming than in Prismatic. However using proper settings it's close to 70-85% of fluidity I see when using Prismatic.

import hid
import socket
from lib27gn950 import *
from time import sleep
monitors = find_monitors()
def rgb_to_hex_safe(rgb):
    r, g, b = rgb
    if r == 0 : r = 1
    if g == 0 : g = 1
    if b == 0 : b = 1
    return '%02x%02x%02x' % (r,g,b)

UDP_IP = "127.0.0.1"
UDP_PORT = 5568

sock = socket.socket(socket.AF_INET, # Internet
                     socket.SOCK_DGRAM) # UDP
sock.bind((UDP_IP, UDP_PORT))

with hid.Device(path=monitors[0]['path']) as dev:
    send_command(control_commands['color_video_sync'], dev)

while True:
    data, addr = sock.recvfrom(1024)
    i = 0
    colors = []
    for ch in data:
        if i%3 == 0:
            r = ch
        elif i%3 == 1:
            g = ch
        elif i%3 == 2:
            b = ch
            colors.insert(i,rgb_to_hex_safe((r,g,b)))
        i+=1
    
    #sleep(0.01)
    with hid.Device(path=monitors[0]['path']) as dev:
        send_video_sync_data(colors, dev)

[mike-def]

Hello guys, I just found this awesome tool, I tried it under Ubuntu for my LG 27gn950 and it works perfectly! Thank you so much for this!
Just wondering - do you have any luck with implementing a video sync?

[koshisan]

Hi! Not sure how active this still is, but I am still using it because of my multi monitor setup and the ability to script it...

I wonder if it would be quicker/easier to implement the WLED sync protocol? https://kno.wled.ge/interfaces/udp-notifier/ This way we could get syncing to work without doing the heavy lifting like reading the screen contents on multiple platforms...