Skip to content

Removing relative paths and adding a test script to spot them#1780

Open
sbAsma wants to merge 14 commits intoecmwf:developfrom
sbAsma:sbAsma/develop/1472-relative-paths
Open

Removing relative paths and adding a test script to spot them#1780
sbAsma wants to merge 14 commits intoecmwf:developfrom
sbAsma:sbAsma/develop/1472-relative-paths

Conversation

@sbAsma
Copy link
Contributor

@sbAsma sbAsma commented Feb 3, 2026

Description

Issue Number

Closes #1472

Is this PR a draft? Mark it as draft.

Checklist before asking for review

  • I have performed a self-review of my code
  • My changes comply with basic sanity checks:
    • I have fixed formatting issues with ./scripts/actions.sh lint
    • I have run unit tests with ./scripts/actions.sh unit-test
    • I have documented my code and I have updated the docstrings.
    • I have added unit tests, if relevant
  • I have tried my changes with data and code:
    • I have run the integration tests with ./scripts/actions.sh integration-test
    • (bigger changes) I have run a full training and I have written in the comment the run_id(s): launch-slurm.py --time 60
    • (bigger changes and experiments) I have shared a hegdedoc in the github issue with all the configurations and runs for this experiments
  • I have informed and aligned with people impacted by my change:
    • for config changes: the MatterMost channels and/or a design doc
    • for changes of dependencies: the MatterMost software development channel

@sbAsma
Copy link
Contributor Author

sbAsma commented Feb 3, 2026

Hi Simon, I am unsure how to handle these paths:

E           Relative path violations found:
E           
E           
E           
E             packages/dashboard/data_overview.py:18
E               Match: "../../stac/jsons
E               Line: stac_dir = (current_dir / "../../stac/jsons").resolve()
E           
E           
E             src/weathergen/train/lr_scheduler.py:334
E               Match: "./plots/lr_schedule.png
E               Line: plt.savefig("./plots/lr_schedule.png")
E           
E             src/weathergen/utils/plot_training.py:25
E               Match: "./config/runs_plot_train.yml
E               Line: DEFAULT_RUN_FILE = Path("./config/runs_plot_train.yml")
E           
E             src/weathergen/utils/plot_training.py:153
E               Match: "./model
E               Line: def get_stream_names(run_id: str, model_path: Path | None = "./model"):
E           
E             src/weathergen/utils/plot_training.py:597
E               Match: "./plots/
E               Line: "-o", "--output_dir", default="./plots/", type=Path, help="Directory where plots are saved"

@jpolz , did you work on the plot_training.py script? Do you have any suggestions?

sbAsma and others added 10 commits February 6, 2026 13:42
Replace relative path handling with _get_shared_wg_path() across
evaluation and export modules. Ensures models, results, and plots
are written to correct shared workspace directories.

Fixes path resolution for inference and evaluation on shared storage.
The code was attempting to log and raise an exception using self.fname_zarr before it was assigned, causing an AttributeError when the Zarr file doesn't exist. Changed to use the local fname_zarr variable instead.
@sbAsma sbAsma marked this pull request as ready for review February 10, 2026 17:09
@sbAsma
Copy link
Contributor Author

sbAsma commented Feb 11, 2026

Hi @iluise thank you for taking the time to review the PR. I reverted the changes, please let me know if there is something I missed, or other changes to add.

Copy link
Collaborator

@iluise iluise left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for reverting the paths. can you change one last thing? Also, I've seen that Till was suggesting to use another existing method to get the paths in another PR. Can you please implement that here as well?

@sbAsma
Copy link
Contributor Author

sbAsma commented Feb 12, 2026

Thanks for reverting the paths. can you change one last thing? Also, I've seen that Till was suggesting to use another existing method to get the paths in another PR. Can you please implement that here as well?

Hi @iluise I removed the changes and applied the method till mentioned where applicable ^^

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

Replace usage of relative pathes (./results, ./models, ...) with shared_directory from private config

2 participants