Notebook restructuring

[martin-springer]

This pull request introduces significant improvements to the project’s development workflow, documentation, and automation. The main themes are: migration of notebooks and scripts to a new tutorials directory structure, enhancements to CI/CD pipelines (including deployment and publishing workflows), and improved contributor documentation. These changes streamline maintenance, clarify contribution processes, and ensure compatibility with modern tooling.

Directory and Notebook Structure Migration:

• All references to tutorials_and_tools are migrated to a new, cleaner tutorials directory structure across CI/CD workflows, pre-commit config, and environment setup, including notebook paths and requirements files. [1] [2] [3] [4] [5] [6] [7]
• Notebook lists in CI workflows (nbval.yaml, testbook.yaml) are updated to match the new directory and naming conventions, improving organization and maintainability. [1] [2]

CI/CD Workflow Enhancements:

• The GitHub Actions workflow for deploying Jupyter Books now supports both main and development branches, and switches to the peaceiris/actions-gh-pages action for more flexible branch previews and artifact deployment. [1] [2] [3]
• The flake8 workflow is refined to exclude scripts in tutorials/**/scripts/ and tools/**/scripts/ from linting, improving performance and relevance. [1] [2]
• The PyPI publishing workflow is updated to publish on all tags, with a separate job for TestPyPI that runs for every tag and a stricter filter for production PyPI releases (excluding pre-releases). Fetch-depth is set to 0 for correct versioning. [1] [2] [3]

Contributor and Documentation Improvements:

• A new CONTRIBUTING.md provides a clear, concise quickstart guide for developers, with links to detailed documentation, database contribution instructions, and pre-commit usage.
• Minor documentation and metadata updates, such as clarifying the branch-to-version mapping in .readthedocs.yaml and updating publication links in README.md. [1] [2]

Other Maintenance:

• License file copyright year is updated for accuracy.

These changes collectively modernize the project structure, improve automation, and make contributing easier for new and existing developers.This pull request reorganizes the project’s notebook and tool directories, updates configuration files to reflect the new structure, and improves the notebook testing and formatting workflows. The main changes include moving notebooks from tutorials_and_tools to a new tutorials directory, introducing a dedicated tools directory for standalone computational tools, updating workflow and pre-commit configurations, and adding support for notebook/script synchronization and formatting.

Directory and Notebook Reorganization:

• All tutorial notebooks have been moved from tutorials_and_tools/tutorials_and_tools/ to a new, cleaner tutorials/ directory, and standalone computational tools have been moved to a new tools/ directory. Workflow and configuration files have been updated to reference these new locations. [1] [2] [3] [4] [5]

Workflow and Environment Updates:

• GitHub Actions workflows for notebook validation (nbval.yaml) and testing (testbook.yaml) have been updated to use the new tutorials/requirements.txt and reference the reorganized notebook paths. [1] [2]

Pre-commit and Formatting Improvements:

• The .pre-commit-config.yaml file now excludes the new tutorials and tools directories from certain hooks, and adds a new hook for jupytext to sync notebooks with scripts and apply black formatting.
• A new jupytext.toml configuration file is added to define notebook/script synchronization and metadata filtering.

Tool Documentation and Dependency Updates:

• Added a new tools/README.md with documentation for each standalone tool and their requirements.
• The pyproject.toml dependencies now include jupytext for notebook/script synchronization.

Notebook Code Style Consistency:

• Notebooks in the new tools directory have been updated for consistent code style, including formatting, string quoting, and minor code cleanups. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14]

Please include a summary of the changes and the related issue. Please also include relevant motivation and context. List any dependencies that are required for this change.

Issue ticket number and link

Fixes # (issue)

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

Checklist before requesting a review

• I have performed a self-review of my code
• Code changes are covered by tests.
• Code changes have been evaluated for compatibility/integration with Scenario analysis (for future PRs)
• Code changes have been evaluated for compatibility/integration with geospatial autotemplating (for future PRs)
• New functions added to init.py
• API.rst is up to date, along with other sphinx docs pages
• Example notebooks are rerun and differences in results scrutinized
• What's new changelog has been updated in the docs

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.67%. Comparing base (51172fd) to head (d8b4c03).
⚠️ Report is 341 commits behind head on development.

Additional details and impacted files

@@               Coverage Diff               @@
##           development     #302      +/-   ##
===============================================
+ Coverage        71.66%   74.67%   +3.01%     
===============================================
  Files               40       41       +1     
  Lines             4524     4813     +289     
===============================================
+ Hits              3242     3594     +352     
+ Misses            1282     1219      -63     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[RDaxini]

@martin-springer I resolved conflicts, removing the pysam notebook (#252) and retaining the modified nbval and 05 advanced pysam notebook from this pr