Removing datasets

[pastewka]

While not a common operation, it would sometimes be useful to easily remove datasets. As far as I understand there are currently no provisions for this and it would need to be supported by the storage broker.

Note that for disk storage one can easily remove the directory, but for other storage brokers (in particular S3) it is less straightforward as the registration key is not stored in the same location as the dataset.

[tjelvar-olsson]

I understand what you want and I have implemented something to do it previously. See:

https://github.com/jic-dtool/dtool-s3/blob/master/remove_dataset/remove_dataset.py

I hesitate adding "remove" functionality to the core API and CLI. The more difficult it is to accidentally delete data the better. However, I think this is a topic that it would be worth discussing in more detail.

[pastewka]

This is great, I hadn't seen this. I just wrote a shell script for this (that uses the aws cli).

It is not urgent but may be useful at some point to discuss whether it is worth putting support for this in the storage broker.