Plans for "hdf5_plugins" repo and releases

[bmribler]

Question for clarification regarding future plans for the hdf5_plugins repository and releases thereof

1. Is the ZFP filter (H5Z-ZFP) kept up to date there with the main Github repo for H5Z-ZFP?
2. Does the fact that ZFP appears to be in the “maintained” category mean that users will get it automatically with an hdf5_plugins release?
3. Do you plan on making pre-built binaries of “hdf5_plugins” as we see, for example, for hdf5-1.12?

Will HDF5 find those plugins automatically, or does the user still need to point to them using HDF5_PLUGIN_PATH and LD_LIBRARY_PATH?
Is the underlying compressor library also bundled?

For internal use only:
SUPPORT-2157

[byrnHDF]

1. and 2. Until an official policy comes out, the answer is yes.

2. The hdf5 binaries include the plugins but there are also separate plugins only binaries that match.
   By the way, I'm unable to download HDF5 from the binary download page (https://www.hdfgroup.org/download-hdf5/). Even though I have an HDF5 account, after logging in the download buttons show "Access denied" with no further explanation. Is that to be expected?

The latest release download pages are available at www.hdfgroup.org.
There are two locations for downloads, the github releases page for hdf5; https://github.com/HDFGroup/hdf5/releases

and the github releases page for hdf5_plugins; https://github.com/HDFGroup/hdf5_plugins/releases

The last two releases and snapshots should be accessible.

Long term storage of releases are on the support site; https://support.hdfgroup.org/releases/v1_14/index.html and https://support.hdfgroup.org/releases/v2_0/index.html (soon)

Will HDF5 find those plugins automatically, or does the user still need to point to them using HDF5_PLUGIN_PATH and LD_LIBRARY_PATH?
A> The binaries are installed to the default location under the hdf5/lib folder - plugin.
But does the user still need to point HDF5_PLUGIN_PATH to this folder or will the library automatically search for them there? Many users likely will not even know how to set HDF5_PLUGIN_PATH (assuming they even know of this feature) or find out where the library plugins are installed.
As long as HDF5_PLUGIN_PATH is set to the same folder as where the plugins are located, no the user does not need to set it. The HDF binaries default is the published default. LD_LIBRARY_PATH (and windows PATH) might still need to be set.

Obviously, there are always exceptions if the defaults are changed in one place and not the others. There are multiple ways to affect the PLUGIN_PATH, within a program and ENV VARS, which can be used after installation.

[byrnHDF]

I assume the "plugins release page" is https://github.com/HDFGroup/hdf5_plugins? Although I appreciate that the plugins are bundled with HDF5 binaries, I suspect many users use package managers to install HDF5. For instance, on my Mac I've used Homebrew and Spack to install HDF5; on RedHat Linux, I used yum. Those packages unfortunately all lack plugins. Similarly, the LLNL compute clusters have HDF5 installations that lack plugins. I don't know to what extent HDF Group is involved in packaging, but is there something that can be done to encourage package creators to include plugins? By the way, I'm unable to download HDF5 from the binary download page (https://www.hdfgroup.org/download-hdf5/). Even though I have an HDF5 account, after logging in the download buttons show "Access denied" with no further explanation. Is that to be expected?

They are also available from the hdf5_plugins release page at the same github repo.

[byrnHDF]

But does the user still need to point HDF5_PLUGIN_PATH to this folder or will the library automatically search for them there? Many users likely will not even know how to set HDF5_PLUGIN_PATH (assuming they even know of this feature) or find out where the library plugins are installed.

As long as HDF5_PLUGIN_PATH is set to the same folder as where the plugins are located, no the user does not need to set it. The HDF binaries default is the published default. LD_LIBRARY_PATH (and windows PATH) might still need to be set. The documentation should explain this HDF5_PLUGIN_PATH / and how to alter it.

[bmaranville]

First of all, thanks so much for maintaining this library! I am using it to build plugins for Webassembly (h5wasm-plugins).

For manipulating the plugin path, I would like to point out that there are HDF5 API functions for adding/removing paths to the list of searched paths. One of them, H5PLinsert is exposed in h5wasm e.g. here:

int insert_plugin_search_path(const std::string search_path_string, unsigned int index)
{
    const char *search_path = search_path_string.c_str();
    herr_t status = H5PLinsert(search_path, index);
    return (int)status;
}

That function is called by the separate h5wasm-plugins package to add a search path for nodejs users that points to the plugins in the package: https://github.com/h5wasm/h5wasm-plugins/blob/v0.1.0/index.mjs#L39