Add scheduler and background task support

[m4tx]

It's common nowadays for web services to do some processing in the background, e.g. video converting, talking to external services, etc. We need some mechanism that would allow us to spin such tasks in a way that would be picked up by some external workers (with an option to run workers inside the same binary for easier development and some simpler deployment cases). We should support some industry-standard way to create a queue of such tasks, e.g. RabbitMQ.

When we have background tasks, a scheduler that would allow running background tasks at given dates and times will probably be useful, too.

When we have all of that, the tasks should be displayed in the admin panel along with their statuses (queued, running, succeeded, failed) and some tools to control them (pause/cancel, etc.).

[wheregmis]

I think instead of integrating directly to some kind of queue like rabbitmq, kafka, redis. Integrating it with something like https://github.com/geofmureithi/apalis might be more feasible.

[yonas]

loco.rs used https://github.com/film42/sidekiq-rs , then rewrote their own.

[xerion12]

hey @m4tx, background jobs and scheduling would make the framework much more complete. i’ve worked before with async task queues and schedulers in rust using redis + apalis, and also custom setups with tokio and cron like jobs. i’d be happy to help design or implement a flexible task system here, maybe start with a trait-based job queue and an in process worker, then add integrations like rabbitmq or redis later.