Skip to content

Releases: climate-resource/input4mips_validation

v0.20.1

01 Oct 14:36
@znichollscr znichollscr
v0.20.1
d200615

Choose a tag to compare

View all tags
v0.20.1 Latest
Latest

Announcements

  • Fixed inference of daily time steps (previously this would fail if data was lazy loaded as subtraction of cftime with xarray doesn't work properly if the data is a dask array)

Changelog

🐛 Bug Fixes

  • Fixed inference of daily time steps (previously this would fail if data was lazy loaded as subtraction of cftime with xarray doesn't work properly if the data is a dask array) (#121)

🔧 Trivial/Internal Changes

Changes

Assets 4
Loading

v0.20.0

04 Sep 20:32
@znichollscr znichollscr
v0.20.0
41a3776

Choose a tag to compare

View all tags
v0.20.0

Announcements

  • the validate-tree CLI will now work even if there are climatology files in the tree
  • we suppport the author field in source ID information

Changelog

🆕 Features

  • Added [input4mips_validation.cvs.author.Author][], [input4mips_validation.validation.email][] and [input4mips_validation.validation.orcid][] to support the inclusion of parsable author information in the source ID CVs. (#116)

🎉 Improvements

  • Added the author attribute to [input4mips_validation.cvs.source_id.SourceIDValues][] to support the inclusion of parsable author information in the source ID CVs. (#116)

🐛 Bug Fixes

  • Fixed the validate-tree CLI (and associated functionality) so that it handles climatology files correctly (#119)

🔧 Trivial/Internal Changes

Changes

Loading

v0.19.1

08 Feb 13:31
@znichollscr znichollscr
v0.19.1
c3393fb

Choose a tag to compare

View all tags
v0.19.1

Announcements

  • Improved the ability to handle data sets with region information, full details below ⬇️

Changelog

🎉 Improvements

  • Added the ability to handle data sets with string variables, like region.
    The key logic improvement is in [XRVariableProcessorLike.get_ds_variables][input4mips_validation.xarray_helpers.variables.XRVariableProcessorLike.get_ds_variables], which now excludes any non-number variables from the dataset's variables. (#113)

Changes

Loading

v0.19.0

26 Dec 14:08
@znichollscr znichollscr
v0.19.0
d6adf11

Choose a tag to compare

View all tags
v0.19.0

Announcements

  • Just cleaning up an unintended part of the API

Changelog

⚠️ Breaking Changes

  • Removed input4mips_validation.database.creation.create_db_file_entry_with_logging, this function was never meant to be released. (#111)

Changes

Loading

v0.18.0

23 Dec 11:18
@znichollscr znichollscr
v0.18.0
5cd4d36

Choose a tag to compare

View all tags
v0.18.0

Announcements

  • We have improved our multiprocessing support and the level of control given to the user. This means that multiprocessing should now work on all systems, including logging on non-windows systems. On windows we don't think the logging will come through properly yet (follow #108 for progress).

Changelog

⚠️ Breaking Changes

  • Changed the interface to [input4mips_validation.database.creation.create_db_file_entry_with_logging][input4mips_validation.database.creation.create_db_file_entry_with_logging] and [input4mips_validation.validation.database.database_file_entry_is_valid][input4mips_validation.validation.database.database_file_entry_is_valid].
    Both of these functions were intended for internal use, hence this should hopefully cause minor issues for users. (#106)
  • Removed (the mostly internal interfaces) input4mips_validation.logging_config.serialise_logging_config and input4mips_validation.logging_config.deserialise_logging_config.
    We don't think these are in general need, although it is still being figured out (see #108). (#107)

🆕 Features

  • Added the --mp-context-id option to the command-line interfaces that support multiprocessing to give the user more control of how the multiprocessing is done.
    This is then passed down the stack to [input4mips_validation.parallelisation.run_parallel][input4mips_validation.parallelisation.run_parallel]. (#107)

🎉 Improvements

  • Added [input4mips_validation.parallelisation][input4mips_validation.parallelisation] to assist with simplifying parallelisation, mainly for internal use. (#106)
  • Added xr_load_kwargs to [input4mips_validation.xarray_helpers.iris.ds_from_iris_cubes][input4mips_validation.xarray_helpers.iris.ds_from_iris_cubes].
    By default, this sets things up so the user doesn't get a thousand warnings about using cftime rather than numpy time objects
    (but the user can pass the argument to change this however they wish). (#107)

Changes

Loading

v0.17.1

22 Dec 16:08
@znichollscr znichollscr
v0.17.1
0f702ad

Choose a tag to compare

View all tags
v0.17.1

Announcements

  • A bug fix release, see below

Changelog

🐛 Bug Fixes

  • Fixed up addition of bounds when using climatology data (time bounds are no longer added). (#104)

Changes

Loading

v0.17.0

22 Dec 15:20
@znichollscr znichollscr
v0.17.0
4e2e8ab

Choose a tag to compare

View all tags
v0.17.0

Announcements

  • We now support monthly climatology datasets
  • There are minor breaking changes, see below

Changelog

⚠️ Breaking Changes

    • input4mips_validation.inference.from_data.infer_time_start_time_end renamed to [input4mips_validation.inference.from_data.infer_time_start_time_end_for_filename][input4mips_validation.inference.from_data.infer_time_start_time_end_for_filename].
    • input4mips_validation.inference.from_data.create_time_range renamed to [input4mips_validation.inference.from_data.create_time_range_for_filename][input4mips_validation.inference.from_data.create_time_range_for_filename].

    (#103)

🆕 Features

  • Added [input4mips_validation.validation.external_variables.validate_external_variables][input4mips_validation.validation.external_variables.validate_external_variables]. (#98)
  • Added [input4mips_validation.validation.frequency.validate_frequency][input4mips_validation.validation.frequency.validate_frequency]. (#100)
  • Added support for data that represents a monthly climatology. (#103)

🎉 Improvements

  • Split [input4mips_validation.validation.allowed_characters][input4mips_validation.validation.allowed_characters] into its own module. (#98)
  • Added [input4mips_validation.inference.from_data.BoundsInfo][input4mips_validation.inference.from_data.BoundsInfo] and propagated it throughout the stack, to make it easier to specify how the bounds variables should be handled. (#100)

🐛 Bug Fixes

  • Fixed the behaviour of [input4mips_validation.cvs.loading_raw.get_raw_cvs_loader][input4mips_validation.cvs.loading_raw.get_raw_cvs_loader]
    (and therefore [input4mips_validation.cvs.loading.load_cvs][input4mips_validation.cvs.loading.load_cvs])
    when the INPUT4MIPS_VALIDATION_CV_SOURCE_FORCE_DOWNLOAD environment variable was set. (#101)

🔧 Trivial/Internal Changes

Changes

Loading

v0.16.0

16 Dec 21:18
@znichollscr znichollscr
v0.16.0
56751d8

Choose a tag to compare

View all tags
v0.16.0

Announcements

This release significantly improves the API and the coverage of the checks provided. It also includes a number of breaking changes, albeit mainly to parts of the API that are not used very much. See the breaking changes for details.

The CLI is also much thiner, with a Python equivalent for each part of the API, should it be needed.

Changelog

⚠️ Breaking Changes

  • Renamed bnds_coord_indicator to bnds_coord_indicators throughout.
    This breaks the API, albeit it is a rarely used part of the API
    and the defaults are sensible.
    Wherever bnds_coord_indicator was used, it now needs to be replaced with a set of strings, e.g. {"bnds"}.
    If using the command-line option, then --bnds-coord-indicator 'bnds'
    would be replaced with --bnds-coord-indicators 'bnds'
    and there is now also the possibility to provide multiple values
    with e.g. --bnds-coord-indicators 'bnds;bounds'. (#87)

    • Removed validate_database_file_entry, validate_ds_to_write_to_disk, validate_file and validate_tree following scheduled deprecation. (#88)

  • Simplified the API in many places.
    This was an attempt to group things which are passed through
    and to make the injectable behaviour clearer throughout.
    Specific changes:

    • Interfaces that now take a [FrequencyMetadataKeys][input4mips_validation.inference.from_data.FrequencyMetadataKeys] rather than frequency_metadata_key and no_time_axis_frequency:
      • [DataReferenceSyntax.validate_file_written_according_to_drs][input4mips_validation.cvs.drs.DataReferenceSyntax.validate_file_written_according_to_drs]
      • [Input4MIPsDatabaseEntryFile.from_file][input4mips_validation.database.database.Input4MIPsDatabaseEntryFile.from_file]
      • [create_db_file_entries][input4mips_validation.database.creation.create_db_file_entries]
      • [get_validate_database_file_entry_result][input4mips_validation.validation.database.get_validate_database_file_entry_result]
    • Interfaces that now take a [XRVariableProcessorLike][input4mips_validation.xarray_helpers.variables.XRVariableProcessorLike] rather than bnds_coord_indicators:
      • [Input4MIPsDataset.write][input4mips_validation.dataset.dataset.Input4MIPsDataset.write]
      • [ds_from_iris_cubes][input4mips_validation.xarray_helpers.iris.ds_from_iris_cubes]
      • [get_ds_to_write_to_disk_validation_result][input4mips_validation.validation.datasets_to_write_to_disk.get_ds_to_write_to_disk_validation_result]
      • [get_validate_file_result][input4mips_validation.validation.file.get_validate_file_result]
    • Interfaces that now take both a [FrequencyMetadataKeys][input4mips_validation.inference.from_data.FrequencyMetadataKeys] rather than frequency_metadata_key and no_time_axis_frequency and a [XRVariableProcessorLike][input4mips_validation.xarray_helpers.variables.XRVariableProcessorLike] rather than bnds_coord_indicators:
      • [get_validate_tree_result][input4mips_validation.validation.tree.get_validate_tree_result]
      • [validate_database_entries][input4mips_validation.validation.database.validate_database_entries]

    (#89)

🆕 Features

  • Added v6.6.0 of the input4MIPs CVs as a known CV source. (#84)

  • Added [input4mips_validation.testing.get_valid_out_path_and_disk_ready_ds][input4mips_validation.testing.get_valid_out_path_and_disk_ready_ds]. (#85)

  • Added the [xarray_helpers.variables][input4mips_validation.xarray_helpers.variables] module
    to help with extracting the variables from an xarray object. (#87)

    • Added [FrequencyMetadataKeys][input4mips_validation.inference.from_data.FrequencyMetadataKeys]
    • Added [XRVariableHelper][input4mips_validation.xarray_helpers.variables.XRVariableHelper]

    (#89)

  • Added Python-equivalent APIs for our CLI commands, see
    [validate_file][input4mips_validation.cli.validate_file],
    [validate_tree][input4mips_validation.cli.validate_tree],
    [db_create][input4mips_validation.cli.db.db_create],
    [db_add_tree][input4mips_validation.cli.db.db_add_tree]
    and [db_validate][input4mips_validation.cli.db.db_validate]. (#90)

  • Added validation of the "Conventions" attribute via [input4mips_validation.validation.Conventions][input4mips_validation.validation.Conventions] (#91)

  • Added [input4mips_validation.cvs.Input4MIPsCVs.validate_activity_id][input4mips_validation.cvs.Input4MIPsCVs.validate_activity_id] and [input4mips_validation.cvs.exceptions][input4mips_validation.cvs.exceptions]. (#92)

  • Added [input4mips_validation.cvs.Input4MIPsCVs.validate_source_version][input4mips_validation.cvs.Input4MIPsCVs.validate_source_version] and [input4mips_validation.cvs.exceptions.ValueInconsistentWithCVsError][input4mips_validation.cvs.exceptions.ValueInconsistentWithCVsError]. (#93)

  • Added [input4mips_validation.cvs.Input4MIPsCVs.validate_contact][input4mips_validation.cvs.Input4MIPsCVs.validate_contact]. (#94)

  • Added [input4mips_validation.validation.comment.validate_comment][input4mips_validation.validation.comment.validate_comment]. (#95)

🎉 Improvements

  • Added headers to the default pooch downloader to workaround readthedocs/readthedocs.org#11763.
    The downloader is injectable, so this behaviour can be overridden if desired. (#84)
  • If a required attribute is missing during validation, we now raise the more explicit [MissingAttributeError][input4mips_validation.validation.exceptions.MissingAttributeError], rather than a KeyError. (#86)

📚 Improved Documentation

  • Updated the docs to use v6.6.0 of the CVs rather than a specific SHA. (#84)

🔧 Trivial/Internal Changes

Changes

Loading

v0.15.0

11 Nov 16:32
@znichollscr znichollscr
v0.15.0
19a32ad

Choose a tag to compare

View all tags
v0.15.0

Announcements

  • We have updated the package to be more in line with the DRS description. This is a breaking change. For full details, please see below.

Changelog

⚠️ Breaking Changes

  • Updated to better be in line with the DRS description.
    The DRS makes no comment on the values in the directory names,
    hence we now make no assumptions and put no requirements on characters used in directory names.
    The restriction to only a-z, A-Z, 0-9 and the hyphen plus special use of the underscore
    is now only applied to file names.
    As a result, files with variables that have underscores in their names
    are now written with underscores in their directories, but not in their file name.
    For example,
    .../mon/mole-fraction-of-methane-in-air/gn/.../mole-fraction-of-methane-in-air_input4MIPs_GHGConcentrations_CMIP_CR-CMIP-0-2-0_gn_200001-201012.nc
    is now written as
    .../mon/mole_fraction_of_methane_in_air/gn/.../mole-fraction-of-methane-in-air_input4MIPs_GHGConcentrations_CMIP_CR-CMIP-0-2-0_gn_200001-201012.nc. (#83)

🐛 Bug Fixes

  • Fixed a bug in
    [validate_file_written_according_to_drs][input4mips_validation.cvs.drs.DataReferenceSyntax.validate_file_written_according_to_drs]
    which caused it to break if the file contained metadata which was not a string. (#83)

🔧 Trivial/Internal Changes

Changes

Loading

v0.14.0

05 Nov 08:40
@znichollscr znichollscr
v0.14.0
e3a445e

Choose a tag to compare

View all tags
v0.14.0

Announcements

  • We have improved the API, particularly for writing data to disk. This gives you much more control over writing and the ability to make changes at multiple steps along the way (which can be necessary, given the complexity of the netCDF conversion). However, this also introduced quite a number of breaking changes. For details, see below.

Changelog

⚠️ Breaking Changes

    • The function signatures for [input4mips_validation.dataset.Input4MIPsDataset.from_data_producer_minimum_information][]
      and [input4mips_validation.dataset.Input4MIPsDataset.from_data_producer_minimum_information_multiple_variable][]
      have now been simplified to use an injectable [prepare_func][input4mips_validation.dataset.dataset.PrepareFuncLike][],
      the default value of which is [input4mips_validation.dataset.dataset.prepare_ds_and_get_frequency][].
      This makes the API simpler and gives the user more control.
      However, it is a breaking change.
    • We removed fine-grained control of writing to disk when using [Dataset.write][input4mips_validation.dataset.Input4MIPsDataset.write]
      and [input4mips_validation.io.write_ds_to_disk][].
      Instead, use [Dataset.get_out_path_and_disk_ready_dataset][input4mips_validation.dataset.Input4MIPsDataset.get_out_path_and_disk_ready_dataset]
      or [input4mips_validation.io.prepare_out_path_and_cubes][] then write using you own writing function
      (typically [iris.save][] instead).
    • Renamed bounds_dim to bounds_indicator in [input4mips_validation.dataset.dataset.handle_ds_standard_long_names][].

    (#80)

🆕 Features

    • Added [Dataset.get_out_path_and_disk_ready_dataset][input4mips_validation.dataset.Input4MIPsDataset.get_out_path_and_disk_ready_dataset]
      and [input4mips_validation.io.prepare_out_path_and_cubes][]
      to permit finer-grained control of file preparation and writing.
    • Added [input4mips_validation.dataset.dataset.add_bounds][].

    (#80)

🐛 Bug Fixes

  • Fixed up passing of input arguments to underlying functions in input4mips-validation validate-tree (#75)
  • [input4mips_validation.cvs.load_cvs][] now supports variables of type Path for the cv_source argument.
    As part of this fix, [input4mips_validation.cvs.loading_raw.get_raw_cvs_loader][] now also supports variables of type Path for the cv_source argument. (#80)

🔧 Trivial/Internal Changes

Changes

Loading