Option to skip processing if target file(s) exists #92
Description
I am processing a lot of files to one minute slots with a custom script which uses the setup logic
gridder, glm_filenames, start_time, end_time, grid_kwargs = grid_setup(args)
gridder(glm_filenames, start_time, end_time, **grid_kwargs)as in the make_grids.py script.
For each gridder call I hand over 180 files (so files for one hour). If for whatever reason I have to restart my script currently all files get processed again which is unfortunate. Ideally I want to skip already processed files to save time.
It would be nice if there was an option for the gridder to skip processing if the files already exist or if there was some helper function which given the files to process and the gridder kwargs would return a list of the target file(s) pathes/names so the user can implement his/her own skipping logic.
On first sight I could not find a function like this or the code which is responsible for constructing the output files pathes/names.