Domain throttling #4
Description
Some domains block spiders when a lot of requests are being send to the server, to prevent this implement throttling on domain-level.
{{ message }}
Some domains block spiders when a lot of requests are being send to the server, to prevent this implement throttling on domain-level.