-
Notifications
You must be signed in to change notification settings - Fork 23
Description
Background
The transfer session model tracks both client and server FSICs within JSON database fields for a directional sync. As time has gone on, these fields have begun to represent a significant amount of data. The data within these fields are then used to generate complex query filters against other tables within the database when queuing data for a sync.
Summary
As someone concerned with performance and scaling of Morango, I'd like to move these FSIC fields to a new, dedicated table, thereby reducing the size of the transfer session table (currently 200+GB). I'd like to take advantage of some performance improvements that can be made by leveraging this data directly within the database when queuing records for syncing.
Deliverables
- A new table
TransferSessionCountershould be created with fields:transfer_sessionFKinstance_idUUIDpartitionnullable text fieldserver_counternullable integerclient_counternullable integer
- The table should be have a unique constraint on
transfer_sessionandinstance_id - API endpoints should serialize the table's data into two separate
client_fsicandserver_fsicfields to maintain compatibility - Queuing operation queries should join against this new table
Notes
Some progress has been made but was done prior to several recent changes to queuing queries 9f919c9