Reloading project during analysis causes permanent failure

[soumyaray]

WHAT: reloading a project that is being cloned/analyzed by worker causes a failure

WHY: multiple possible reasons

• corruption of local gitstore (folder where repos are being cloned could be overwritten/damaged)
  • should lock repos being cloned/analyzed?
• corruption of MongoDB cache (sometimes needs to be reset for failing projects)
  • should wipe mongo cache for projects causing problems?
• @tienyulin other potential reasons?

[tienyulin]

Fixed it, removed an error handler that does not have any further processing.
But reloading means sending a new message to SQS, so worker will process the new one after processing the current one.

[soumyaray]

I think that's fine as long as user doesn't experience a hang or a crash. its possible someone wants to reanalyze a project even 10 seconds after the first time because something changed. But we have to make sure it doesn't cause a conflict on local DB (cloning) or MongDB. I'm guessing (I don't know) that Mongo will take care of simultaneous write attempts. The conflict we are stuck with is if one process deletes the cloned folder to make another clone. Does this sound right or do you think there is something else that can cause an appraisal process to crash?