Improve performance with batching

[theblazehen]

Currently it only uploads one log line with each HTTP request, which does not work well for high velocity logs.

The queued handler could potentially read all the available lines in the queue and upload everything outstanding at once as opposed to popping an item one at a time, and uploading it

[tomashornak]

This would be nice feature, otherwise I'm little bit afraid to use it in production on high-load web servers.
Did you come to any solution @theblazehen ?

[theblazehen]

@tomashornak Afraid not, decided to just log to journald with https://pypi.org/project/systemd-logging/ and use promtail from there, but haven't tested it at scale yet

[nathabit]

@tomashornak Afraid not, decided to just log to journald with https://pypi.org/project/systemd-logging/ and use promtail from there, but haven't tested it at scale yet

How are you propagating unique_labels, tags etc in your solution?

[LuisGMM]

Is there any new about this?

It is true that in an async environment with some load a logger that can batch logs becomes necessary :)