diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index da9a89d..aa27210 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -86,6 +86,12 @@ jobs: .log/coverage/htmlcov .log/coverage/coverage.xml + - name: Upload coverage reports to Codecov + if: ${{ matrix.os == 'ubuntu-latest' && matrix.python-version == env.PYTHON_VERSION }} + uses: codecov/codecov-action@v5 + with: + token: ${{ secrets.CODECOV_TOKEN }} + build-distribution: name: Build Package Distribution runs-on: ubuntu-latest diff --git a/.github/workflows/pr-checks.yml b/.github/workflows/pr-checks.yml index 538bd08..b0a46f6 100644 --- a/.github/workflows/pr-checks.yml +++ b/.github/workflows/pr-checks.yml @@ -159,6 +159,15 @@ jobs: .log/coverage/htmlcov .log/coverage/coverage.xml + - name: Upload coverage reports to Codecov + uses: codecov/codecov-action@v5 + with: + token: ${{ secrets.CODECOV_TOKEN }} + override_pr: ${{ github.event.pull_request.number }} + slug: ${{ github.repository }} + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + conflict-check: name: Check for Merge Conflicts runs-on: ubuntu-latest diff --git a/README.md b/README.md index bb460f5..fd8d5a4 100644 --- a/README.md +++ b/README.md @@ -13,6 +13,7 @@ [![Security][security-badge]](https://github.com/Pymetheus/python-project-blueprint/actions/workflows/security.yml) [![CI][ci-badge]](https://github.com/Pymetheus/python-project-blueprint/actions/workflows/ci.yml) [![CD][cd-badge]](https://github.com/Pymetheus/python-project-blueprint/actions/workflows/cd.yml) +[![CD][codecov-badge]](https://codecov.io/github/Pymetheus/python-project-blueprint) [![Bootstrap][bootstrap-badge]](https://github.com/Pymetheus/python-project-blueprint/actions/workflows/bootstrap.yml) [![Template][use-template-badge]](https://github.com/new?template_name=python-project-blueprint&template_owner=Pymetheus) @@ -29,6 +30,7 @@ [security-badge]: https://github.com/Pymetheus/python-project-blueprint/actions/workflows/security.yml/badge.svg [ci-badge]: https://github.com/Pymetheus/python-project-blueprint/actions/workflows/ci.yml/badge.svg [cd-badge]: https://github.com/Pymetheus/python-project-blueprint/actions/workflows/cd.yml/badge.svg +[codecov-badge]: https://codecov.io/github/Pymetheus/python-project-blueprint/graph/badge.svg?token=Y4K5R5F457 [bootstrap-badge]: https://github.com/Pymetheus/python-project-blueprint/actions/workflows/bootstrap.yml/badge.svg [use-template-badge]: https://img.shields.io/badge/Use%20this%20template-006222 @@ -68,6 +70,7 @@ By establishing structure, tooling, and automation upfront, it reduces the need - **Quality & Safety by Default** - Coverage enforcement with `pytest` + - Cloud-based coverage reporting and visualization with `Codecov` - Minimum 80% coverage threshold - Fast linting and formatting via `ruff` - Strict static typing with `mypy` diff --git a/codecov.yml b/codecov.yml new file mode 100644 index 0000000..ab51ad4 --- /dev/null +++ b/codecov.yml @@ -0,0 +1,15 @@ +comment: + layout: "diff, flags, files" + behavior: default + require_changes: false # Change to true for less noise + require_base: true + require_head: true + +coverage: + status: + project: + default: + target: 80% + patch: + default: + informational: true diff --git a/docs/CHECKLIST.md b/docs/CHECKLIST.md index ca0547c..c3a3b16 100644 --- a/docs/CHECKLIST.md +++ b/docs/CHECKLIST.md @@ -21,10 +21,17 @@ Follow this checklist to set up your own Python project with the Python Project ### Snyk Account - [ ] **USER:** Create a [Snyk account](https://app.snyk.io/login) if not already present. - [ ] **USER:** Add `SNYK_TOKEN` as a **new repository secret** under **Settings** → **Secrets and variables** → **Actions** & **Dependabot**. (found under https://app.snyk.io/account → **Auth Token**) -- [ ] **USER:** Add `SNYK_ORG_ID` as a **new repository secret** under **Settings** → **Secrets and variables** → **Actions** & **Dependabot**. (found under https://app.snyk.io/org/ /manage/settings → **Organization ID**) +- [ ] **USER:** Add `SNYK_ORG_ID` as a **new repository secret** under **Settings** → **Secrets and variables** → **Actions** & **Dependabot**. (found under https://app.snyk.io/org/ your-org /manage/settings → **Organization ID**) > **Note:** *If you don't want to use Snyk for security scanning, delete or replace the `snyk-security-scan` job in `.github/workflows/security.yml`.* +### Codecov +- [ ] **USER:** Create a [Codecov account](https://app.codecov.io/login) if not already present. +- [ ] **USER:** Add `CODECOV_TOKEN` as a **new repository secret** under **Settings** → **Secrets and variables** → **Actions** & **Dependabot**. (found under https://app.codecov.io/gh/ your-org / your-repo /new → **Step 3: add token**) +- [ ] **USER:** Install the Codecov App on your GitHub organization and give repository access with `Only select repositories`. + +> **Note:** *If you don't want to use Codecov for PR comments and testing visualization, delete the `codecov.yml` file and the `Upload coverage reports to Codecov` step in `.github/workflows/ci.yml` and `.github/workflows/pr-checks.yml`.* + ### TestPyPI (Optional) - [ ] **USER:** Create a [TestPyPI account](https://test.pypi.org/account/register/) if not already present. - [ ] **USER:** Add a new project under [pending publisher](https://test.pypi.org/manage/account/publishing/) diff --git a/docs/INSTRUCTIONS.md b/docs/INSTRUCTIONS.md index 76419eb..d627c7a 100644 --- a/docs/INSTRUCTIONS.md +++ b/docs/INSTRUCTIONS.md @@ -9,19 +9,19 @@ It reduces setup and automation overhead, allowing teams to focus on application The **Python Project Blueprint** comes with built-in: -| Feature | Description | -|:-----------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------| -| **Project Management** | `pyproject.toml` as the centralized configuration for packaging, tooling, metadata and dependency management. | -| **Config Management** | `pydantic-settings` for type-safe environment variable loading and validation. | -| **Structured Logging** | `structlog` integration for JSON streams (production) and colored console output (development). | -| **Professional Layout** | `src/` directory structure to ensure package integrity and prevent local import conflicts. | -| **Comprehensive Testing** | `pytest` paired with coverage reporting and enforced minimum thresholds for deep visibility into code health. | -| **Static Analysis** | `ruff` for linting/formatting and `mypy` for strict static type checking. | -| **Automated Security** | `Snyk` (OSS) and `bandit` (SAST) integrated to scan for vulnerabilities and leaked secrets. | -| **Standardized Governance** | Automated label synchronization, Issue templates, and Pull Request templates. | -| **Local Automation** | `.pre-commit-config.yaml` for linting, formatting, and security checks. | -| **CI/CD Pipeline** | `pr-checks`, `ci`, `cd`, and `security` GitHub Actions workflows. | -| **Instant Bootstrapping** | A `bootstrap.yml` GitHub Action to rebrand and initialize the repository in seconds. | +| Feature | Description | +|:-----------------------------|:----------------------------------------------------------------------------------------------------------------------------------------| +| **Project Management** | `pyproject.toml` as the centralized configuration for packaging, tooling, metadata and dependency management. | +| **Config Management** | `pydantic-settings` for type-safe environment variable loading and validation. | +| **Structured Logging** | `structlog` integration for JSON streams (production) and colored console output (development). | +| **Professional Layout** | `src/` directory structure to ensure package integrity and prevent local import conflicts. | +| **Comprehensive Testing** | `pytest` and `Codecov` integration with coverage reporting to enforce coverage thresholds and ensure deep visibility into code health. | +| **Static Analysis** | `ruff` for linting/formatting and `mypy` for strict static type checking. | +| **Automated Security** | `Snyk` (OSS) and `bandit` (SAST) integrated to scan for vulnerabilities and leaked secrets. | +| **Standardized Governance** | Automated label synchronization, Issue templates, and Pull Request templates. | +| **Local Automation** | `.pre-commit-config.yaml` for linting, formatting, and security checks. | +| **CI/CD Pipeline** | `pr-checks`, `ci`, `cd`, and `security` GitHub Actions workflows. | +| **Instant Bootstrapping** | A `bootstrap.yml` GitHub Action to rebrand and initialize the repository in seconds. | > **Note on Bootstrapping Process:** > @@ -111,6 +111,7 @@ python-project-blueprint/ ├── .dockerignore # Docker build exclusions ├── .gitignore # Git exclusions ├── .pre-commit-config.yaml # Local linting/security gates +├── codecov.yml # Codecov configuration file ├── LICENSE.md # Usage permissions ├── pyproject.toml # Central configuration hub (Dependencies & Tools) └── README.md # Project landing page @@ -193,6 +194,7 @@ By using these tools, the project ensures that every contribution is vetted befo To scale collaboration and maintain a clean repository, the following tools are integrated: * **Standardized Entry Points:** `ISSUE_TEMPLATE/` and `PULL_REQUEST_TEMPLATE.md` ensure that every **bug report** or **feature request** contains the necessary technical context (logs, environment info, reproduction steps). +* **Automated Quality Gates:** The `pr-checks.yml` workflow uploads test results to `Codecov` to enforce coverage thresholds and provide automated PR comments that visualize testing gaps. * **Automated Labeling (`label-sync.yml`):** Automatically synchronizes repository labels based on a central `labels.yml` file, keeping the project organized and searchable. * **Access Control (`CODEOWNERS`):** Defines which team members are responsible for specific parts of the codebase, automatically assigning them as reviewers for relevant Pull Requests. @@ -248,7 +250,7 @@ This workflow ensures that every contribution meets the project's quality, stabi > > **When:** On Pull Requests to `main` branch. > -> **Focus:** Static analysis (linting/types), unit testing, and metadata validation. +> **Focus:** Static analysis (linting/types), unit testing, coverage reporting, and metadata validation. ````yaml name: PR Checks @@ -269,14 +271,14 @@ jobs: pr-description-check: # Guarantees context for every code change. lint: # Enforces style via Ruff mypy: # Verifies type safety - test: # Executes unit tests and generates coverage artifacts. + test: # Executes unit tests and uploads coverage to Codecov. conflict-check: # Detects merge conflicts early. size-check: # Warns if large files are being detected. pr-checks-summary: # Provides a summary of the previous jobs. pr-labeler: # Automates project board organization. ```` -> **Note:** *Detailed test coverage artifacts are available under the Actions > Artifacts tab for every successful run.* +> **Note:** *Test coverage reports are automatically uploaded to Codecov for visualization. Raw artifacts are also available under the Actions > Artifacts tab.* **PR Guidelines:** This project uses [Conventional Commits](https://www.conventionalcommits.org/) for PR titles. @@ -289,6 +291,8 @@ jobs: - ✅ `feat: add user login` - ❌ `Updated code` (missing type) +**PR Codecov Report:** Provides an automated comment in every PR with a coverage difference and impact analysis. + **PR Types & Labeling:** The repository automatically assigns labels to pull requests based on their title. @@ -315,9 +319,9 @@ The CI process performs a deep validation to ensure that the integrated code is > **Purpose:** Final verification and build preparation. > -> **When:** Pushes to main +> **When:** Pushes to the `main` branch. > -> **Focus:** Extensive testing, distribution builds, and smoke testing +> **Focus:** Extensive testing, distribution builds, and smoke testing. ````yaml name: CI @@ -330,12 +334,12 @@ on: jobs: verify-version: # Preventing deployment collisions. - test: # Executes the complete test suite with coverage. + test: # Executes the complete test suite with coverage and uploads report to Codecov. build-distribution: # Packages the code into a distributable format. smoke-test: # Proves the package is installable and functional. ```` -> **Note:** *Detailed test coverage artifacts are available under the Actions > Artifacts tab for every successful run.* +> **Note:** *Test coverage reports are automatically uploaded to Codecov for visualization. Raw artifacts are also available under the Actions > Artifacts tab.* --- @@ -343,11 +347,11 @@ jobs: This workflow automates the final stage of the software lifecycle: **Delivery**. By triggering only after a completed CI run, it ensures that only fully vetted and smoke-tested code is ever published to the ecosystem. -> **Purpose:** Production deployments +> **Purpose:** Production deployments. > -> **When:** On completed `CI` workflow run +> **When:** On completed `CI` workflow run. > -> **Focus:** Publish package and create release +> **Focus:** Publish package and create release. ````yaml name: CD @@ -372,11 +376,11 @@ jobs: This workflow acts as an automated security audit, scanning your codebase and dependencies for vulnerabilities before any code is merged. It ensures the project remains compliant with basic security standards. -> **Purpose:** Security audits & vulnerability scanning +> **Purpose:** Security audits & vulnerability scanning. > -> **When:** On PRs +> **When:** On PRs. > -> **Focus:** Dependencies, code security, SAST +> **Focus:** Dependencies, code security, SAST. ````yaml name: Security Scan @@ -466,12 +470,12 @@ The `tests/` directory is structured to provide high coverage with minimal frict * **`test_main.py`**, **`test_config.py` & `test_logger.py`**: Baseline tests that ensure the source code is functioning correctly. ### Quality Metrics & Artifacts -The testing workflow is integrated with the `.log/` directory to provide deep insights into code health: +The testing workflow is integrated with the `.log/` directory and `Codecov` to provide deep insights into code health: * **Automated Coverage**: Every test run generates a `.coverage` report, allowing you to see exactly which lines of code are untested. * **Coverage Threshold**: The project enforces a **minimum coverage of 80%**. If code coverage falls below this mark, the `pr-checks` workflow will fail, preventing merges that reduce test quality. -### Running Tests +### Running Tests Locally Pytest is configured via `pyproject.toml` to include coverage reporting and verbose output by default. @@ -480,6 +484,14 @@ Just run: pytest ``` +### Coverage Reporting with Codecov +This project utilizes Codecov for cloud-based coverage analysis. +This service transforms raw test data into actionable insights during the review process via the `codecov.yml` configuration. +* **PR Integration**: Delivers an automated Codecov Report to every PR, featuring coverage deltas and impact analysis. +* **Automated Uploads**: Every time tests are executed in `pr-checks.yml` or `ci.yml`, reports are automatically synced to the Codecov dashboard. +* **Informational Patch**: Provides targeted feedback on the coverage of only the changed lines within a Pull Request. +* **Visualizations**: Offers interactive charts on the Codecov dashboard to identify complex, low-coverage modules. + --- ## Project Management with pyproject.toml