-
Notifications
You must be signed in to change notification settings - Fork 43
Open
Description
Please update the README:
- To outline how the library works.
- To clarify if clustering is supported or not.
After reading the source code:
I guess clustering (running tasks on many machines) is not supported yet, because:
- On startup, all running tasks are forcefully resumed. The code doesn't care that the tasks may be being taken by other machines.
- When taking new tasks, the code only care about newly inserted tasks. It doesn't care about the tasks that have been marked as taken by other machines, but other machines may have died.
To support clustering with "at least once" semantic, these little modifications can be made ("little" compared to the existing great work):
- Add visibility timeout column to the tasks DB table: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html
- At startup and when taking new tasks, use the visibility timeout to figure out if a machine should take over tasks that have been marked as taken.
akhoury, andreas-soroko, vafrcor, focux, AdelUnito and 5 more
Metadata
Metadata
Assignees
Labels
No labels