Govcms websites limits

[koriaf]

Now we determine the GovCMS website by seeing the text govcms (case insensitive) in the X-Generator header.
This approach works so far, but the limit is global: so 2 different such websites won't be crawled simultaneously. It's fine while they share the same hardware but completely pointless in case of separate installations, and makes all govcms websites crawling a pretty time-consuming operations (while they won't be crawled in parallel).
There are some proposed solutions:

• have the static list of govcms websites which can be crawled simultaneously
• have the static list of govcms websites which can't
• determine limits based on some other headers or somehow else understand if these websites are safe to be crawled at same time and these are not

Issue is not the most critical, but nice to be resolved.