Handle Singularity 3.6 and later /tmp staging via automatic %setup fallback #527
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Singularity 3.6 and later disallow staging files to /tmp or /var/tmp via the %files section before the container rootfs exists. Add a selective %setup fallback that copies only the affected entries while preserving existing %files behavior for all other paths. This resolves staging failures introduced by Singularity 3.6 without impacting Docker or multi-stage builds.
Introduce a global configuration flag to enable or disable the automatic %setup fallback for /tmp and /var/tmp destinations on Singularity >= 3.6. The fallback is enabled by default to preserve backward compatibility.
Expose the Singularity tmp fallback configuration via the command-line interface using a standard argparse boolean flag. This allows users to restore the pre-3.6 error behavior when staging files to /tmp or /var/tmp via %files.
Add unit tests covering: - single and multiple /tmp and /var/tmp destinations - nested paths - mixed safe and tmp destinations - disabled fallback error behavior - unchanged Docker behavior - interaction with _exclude_from
Expose the Singularity /tmp fallback behavior through the recipe() API and forward it to global configuration.
Document the Singularity 3.6 and later restriction on staging files to /tmp and /var/tmp via %files, and describe the automatic %setup fallback behavior enabled by default. Also document the copy primitive _exclude_from option, including its rsync-based %setup behavior for selective file exclusion.
Contributor
Author
|
Thanks a lot for the review and for merging this, @samcmill. I really appreciate the feedback and guidance throughout the process! Best! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request Description
This PR resolves build failures when copying files to
/tmpor/var/tmpusing the copy primitive on Singularity / Apptainer 3.6 and later by introducing a selective%setupfallback, enabled by default.The change aligns HPCCM behavior with upstream Singularity restrictions while preserving backward compatibility and minimizing user-facing disruption.
Background
Singularity 3.6+ no longer allows files to be staged into
/tmpor/var/tmpvia the%filessection, as the container root filesystem does not yet exist at that stage. This caused HPCCM recipes using:to fail with a runtime error (see #345).
While
--working-directoryaddresses build-time execution in%post, it does not affect%filesstaging and therefore cannot resolve this issue.What this PR does:
Automatic
%setupfallback (default behavior)When targeting Singularity 3.6+:
Only copy entries whose destination is under
/tmpor/var/tmpare moved into a
%setupblock.Other copy entries remain in
%files, preserving existing semantics.Example output:
%setup mkdir -p ${SINGULARITY_ROOTFS}/tmp cp -a foo ${SINGULARITY_ROOTFS}/tmp/foo %files bar /opt/barConfigurable behavior
A new configuration flag is introduced:
and can be disabled via CLI:
If disabled, HPCCM preserves strict behavior and raises a clear runtime
error on Singularity 3.6+ when
/tmpor/var/tmpappears in%files.References
#345
apptainer/singularity#5848
apptainer/apptainer#490
Author Checklist
pydocmd generate) if any docstrings have been modified