Skip to content

Reduce duplicate building on CI (plus other CI improvements)#3642

Merged
DrPaulSharp merged 13 commits intoSasView:mainfrom
llimeht:tmp/ci-caching
Oct 24, 2025
Merged

Reduce duplicate building on CI (plus other CI improvements)#3642
DrPaulSharp merged 13 commits intoSasView:mainfrom
llimeht:tmp/ci-caching

Conversation

@llimeht
Copy link
Contributor

@llimeht llimeht commented Oct 19, 2025

Description

This PR capitalises on the work to make sasview wheels and make sure that sasview is platform-independent (pure python) project.

  • With these properties now well established, we can just build sasview once on CI as the wheel is identical on all platforms. This saves total CPU time by building once rather than across the entire n CPUs × m interpreters matrix; it also saves total CI time by avoiding building on the slowest platform (Windows).
  • We have been building the wheels for sasdata and sasamodels just once for a while - this PR now takes the opportunity to cache those wheels so that they aren't rebuilt if their git trees are not changed.
  • We can split the "Test" and "Build Installer" phases within the workflow graph; they already ran in parallel but it just always looked wonky to have them inside the same matrix

This PR also uncovers (and fixes) a couple of bugs

  • CI recently started failing on ubuntu-latest because it no longer had the tools to compile pycairo (unfortunately pypi does not have wheels for non-Windows platforms); the necessary build-deps are now included in case pycairo needs to be built. (I wonder if that should be added to a caching step instead) -- edit: this was already separately found and fixed on the release_6.1.1 branch so I dropped these patches in rebasing
  • The build-dependencies for sasview need to include various packages that are imported, otherwise sphinx can't generate the documentation due to ImportErrors. These packages are now added to pyproject.toml and build_tools/requirements-dev.txt.
  • Installation of linux packages was likely installing unneeded packages via the Recommends field, so that is suppressed as is standard for handling build-dependencies; there's also a small win in reducing the belts-and-braces file handling safety inside the container with eatmydata.
  • test the installed version of the module not the unpacked source, verifying that the installation artifacts we send out actually work (this is now possible thanks to move qt gui build to cli.py and remove run.py #3596)
  • one of the test modules was incorrectly using src.sas.… as an import; fixed in this PR

Just for the moment, I've used the AUSAXS downloader to place the AUSAXS library into the source tree; with #3596 merged, we'll run the tests against the installed version rather than the raw checkout, and then the downloader should download into the installed library instead (the relevant code is in get_external_dependencies.py already). And of course once #3634 is merged, then the library will be installed via normal dependency handling and that code gets deleted.

I think this work is ready to review and merge, however... we're tripping over each other with lots of PRs touching the same files. Some file-conflict-resolution and merging is needed to clear the release-6.1.1 backlog (at least) prior to merging this PR (probably #3596 and #3634 as well), but then there's substantial work required to merge all these.

How Has This Been Tested?

CI

Review Checklist:

[if using the editor, use [x] in place of [ ] to check a box]

Documentation (check at least one)

  • There is nothing that needs documenting
  • Documentation changes are in this PR
  • There is an issue open for the documentation (link?)

Installers

  • There is a chance this will affect the installers, if so
    • Windows installer (GH artifact) has been tested (installed and worked)
    • MacOSX installer (GH artifact) has been tested (installed and worked)
    • Wheels installer (GH artifact) has been tested (installed and worked)

Licensing (untick if necessary)

  • The introduced changes comply with SasView license (BSD 3-Clause)

Copy link
Contributor

@DrPaulSharp DrPaulSharp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. I'll merge this now.

@DrPaulSharp DrPaulSharp merged commit 0ed7b39 into SasView:main Oct 24, 2025
15 checks passed
@DrPaulSharp DrPaulSharp mentioned this pull request Oct 24, 2025
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants