Frozen Dataset Object mechanic not working

[SickSmile1]

In the case if object is used as a dataset instead of the class DataSetCreator, the frozen mechanic does not work and the same proto dataset can be frozen multiple times with changes of the dataset in between.

Example to recreate issue:

timestamp = str(time.time())
current_dir = os.getcwd()
data_dir2 = os.path.join(current_dir,timestamp+"_data")

os.makedirs(data_dir2, exist_ok=True)

with open("jupyter_test_file.txt", "w") as f:
    f.write("This is a test file")

new_dataset = dtoolcore.create_proto_dataset(
    name="my-dataset",
    base_uri=data_dir2
)

new_dataset.put_item( "jupyter_test_file.txt", "jupyter_test_file.txt")

readme = yaml.dump(    dict(
        description="For demonstation purposes only",
        project="Today's test project.",
        author="Johannes Laurin Hoermann",
        username="jotelha",
        orcid="0000-0001-5867-695X",
        organization="University of Freiburg",
        program="Haushaltsstelle"
    ))
new_dataset.put_readme(readme)

new_dataset.freeze()

# second file after new_dataset is frozen 
with open("another_jupyter_test_file.txt", "w") as f:
    f.write("This is another test file")

new_dataset.put_item( "another_jupyter_test_file.txt", "another_jupyter_test_file.txt")

new_dataset.freeze() 
# freezing a frozen dataset object doesnt produce an error. 
# the cli confirms the frozen status after the first freeze

[tjelvar-olsson]

I can reproduce the scenario that you are illustrating and I understand that it is less than ideal/confusing.

The short answer is that the Python API is probably more powerful than it should be, and that it is not intended to be used in this way.

The somewhat longer answer is that the state of a dataset is determined by the class, e.g. ProtoDataSet vs DataSet. A DataSet cannot be frozen, whereas a ProtoDataSet can. On top of that there are some guard rails to ensure that one cannot instantiate a ProtoDataSet from a DataSet.

However, in this case the ProtoDataSet has already been instantiated, it just lingers around longer than it should. This is one of the reasons why the DataSetCreator context managers are better than using the raw create_proto_dataset function.

It is worth noting that the state in the admin metadata is updated correctly when freezing the proto dataset, and one could imaging implementing a check on all ProtoDataSet methods (that make modifications) to assert that self._admin_metadata['type'] == 'protodataset'.