dynamic delay calc

[jfroelich]

There are tradeoffs to naively rescheduling with a delay of 0.

When the queue is empty, and will remain empty for some material amount of time, then every reschedule is pointless, and it would be better to reschedule less often.

When the queue is not empty, and appears to be constantly backing up, then every reschedule is valuable, and it would be better to reschedule as frequently as possible to avoid any unnecessary stalling.

When the queue is not empty, it also depends on the average time length. If the queue is pretty much always not empty but its task are long running, then the queue will tend to remain saturated, making most reschedules a waste. If the queue is generally not empty but its tasks are mostly brief, where the queue is generally not saturated all the time, then it is beneficial to poll (reschedule) frequently.

One idea is that instead of hardcoding busyDelay, it is a pluggable function. The library ships with a default function that naively returns a static 0 value. Then users can plugin whatever delay function they want. And I could eventually introduce one with more intelligence.

[jfroelich]

Another related thing here is some kind of long polling equivalent. In one call to drain(), instead of only making one attempt to dequeue, we maybe make a few attempts. Basically retries at getting the queue head all in one go, instead of reschedules. Or something like this, so far this sounds stupid.

[jfroelich]

dynamic polling

• somehow keep a history in order to calculate rate of calls
• if tasks/period rate is low, wait more
• if tasks/period rate is high, wait less

then

• calculate reschedule delay dynamically instead of hardcoding