Skip to content

Updates for Middleman v2#46

Open
marc1uk wants to merge 10 commits intoToolDAQ:mainfrom
marc1uk:mc_locking
Open

Updates for Middleman v2#46
marc1uk wants to merge 10 commits intoToolDAQ:mainfrom
marc1uk:mc_locking

Conversation

@marc1uk
Copy link
Contributor

@marc1uk marc1uk commented Nov 24, 2025 •
edited
Loading

  • in ToolChainConfig templates, split multicast traffic between 3 addresses for ServiceDiscovery, Logging and Monitoring (239.192.1.1,2,3 respectively) but now by default using the same port (5000) although configurable
  • in ServiceDiscovery::MulticastListenThread bind to the multicast address, not INADDR_ANY, as IP_ADD_MEMBERSHIP acts globally so is not sufficient to listen for only this source of traffic.
  • set linger option on zmq sockets in ServiceDiscovery

Services class:

  • fix use of magic number 655355 -> MAX_UDP_PACKET_SIZE
  • convert timestamps from unix ms to time string locally, rather than in the middleman, for better accuracy
  • SendAlarm populates 'alarm' field not 'message' field in database (previously field name was mapped in middleman)
  • make topics 'LOGGING', 'MONITORING' uppercase for consistency
  • SendCalibrationData, SendDeviceConfig, SendRunConfig no longer quote data field for updated middleman
  • response format for insertions is no longer JSON '{"version": val}' and just 'val': update response handling
  • Get functions (GetCalibrationData, GetDeviceConfig, GetRunConfig, GetRunDeviceConfig...) now send SQL, not JSON. Response and API returns are unchanged.
  • GetRunDeviceConfig now works via a single query with subquery rather than two separate queries

ServicesBackend class:

  • split logging and monitoring to different addresses
  • replace AddService with AddPort, RemoveService with RemovePort for updated Utilities class - ports are now named 'db_read' and 'db_write'
  • remove narrowing cast of timeout
  • add mutex locking of socket in SendMulticast function in case calling application is multi-threaded
  • change message id and failure tracking counters to atomics

API changes:

  • fix type of timestamp arguments (unsigned int -> uint64_t)
  • rename SendCalibrationData argument 'device' to 'name'
  • SendTemporaryRootPlot and SendPersistentRootPlot are now SendRootPlotMulticast and SendRootPlotZmq. SendRootPlot defaults to the latter. SendPlotlyPlot always uses the latter.
  • Send*Plot functions now take a lifetime stored in the database. Default lifetime is 5 versions.
  • SQLQuery no longer accepts a database argument
  • GetROOTplot no longer potentially optionally the timestamp of the associated plot.

N.B.
backports required for old middleman:

  • fix type of timestamp arguments (unsigned int -> uint64_t)

Marcus O'Flaherty and others added 5 commits November 24, 2025 10:18
add mutex locking to multicast sockets
3730202
@superkanalysis
Services signature updates
0b54148
@superkanalysis
fix DAQLogging to bind logging multicast listener to the specific gro…
c2942df 
…up, rather than INADDR_ANY, as joining the group is not restricted to the socket (but binding is). Also add linger settings to zmq sockets
@superkanalysis
putting logging and monitoring on different multicast addresses (but …
a1348a7 
…default to the same port)
@superkanalysis
DAQUtilities: whitespace and commented-out printout
e77dbd2 
ServiceDiscovery: add continue such that when there are messages available from the inproc socket, they are all processed before moving to broadcast part. Otherwise this means at most one message is processed per SD broadcast period.
Services: add lifetime argument. conversion from unix ms to time string now done on client not MM. some DB field renames, remove quotes around embedded JSON. Responses with version numbers are just the number, no longer JSON, for reduced traffic. Get requests use SQL directly with json_build_object to have Postgres form JSON directly. Remove option to retrieve timestamp with GetRootPlot and GetPlotlyPlot for simplicity.
ServicesBackend: make counters atomic. Update port advertisement methods.
@marc1uk marc1uk changed the title add mutex locking to multicast sockets Updates for Middleman v2 Jan 21, 2026
@marc1uk
Copy link
Contributor Author

marc1uk commented Feb 1, 2026 •
edited
Loading

linked to ToolDAQ/libDAQInterface#20 and ToolDAQ/WebServer#99

@marc1uk marc1uk mentioned this pull request Feb 1, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@marc1uk @superkanalysis