Add --auto-cleanup option to delete job directory after successful completion

[baminou]

[junjun-zhang]

hmm, @baminou, this may not be as simple as we thought about it earlier. In the situation where all of the tasks were completed by a single executor, it's very easy to do, the executor will just need to delete the job dir. But what if a job's tasks were spread out to multiple executors each on different compute node, not all the executors know the job has been completed, only the executor finished the last task of the job knows it, other executors don't.

[junjun-zhang]

There is a solution. As this is kind of house keeping work for Executor, what we can do is to make executor iterate through its local workdirs for jobs have been executed. If a job has been completed (by query the jt-jess server), the executor can then safely cleanup the workdir for this job.