resolution code race condition for pushing to git in parallel processes #148
Description
Since efd6234 there is a race condition for pushing resolution sets to the forecastbench datasets repo.
If GCP Cloud Run tasks take a similar time to run, there's a chance that two or more tasks may clone the datasets repository at the same time then there's a race condition for pushing.
Potential solutions:
- quick fix: on fail, fetch and push again in a loop
- (preferred) complete fix: store all resolution sets in a bucket then create a GCP Cloud Run job that pushes all resolution sets in one commit.