Merged
Conversation
The project previously existed only as a standalone script without proper packaging infrastructure. This made it difficult to install as a Python package, distribute via PyPI, or integrate into larger projects. Shared GitLab functionality was also duplicated across multiple scripts. Introduce a standard Python package structure with src layout, pyproject.toml configuration, and a shared gitlab_common module. The package structure follows modern Python packaging conventions with setuptools, while the shared module consolidates Git repository detection, project resolution, and error handling logic that was previously scattered or duplicated. This enables installation via pip while maintaining backward compatibility with direct script execution. Signed-off-by: Javier Tia <javier.tia@linaro.org>
GitLab's Generic Package Registry enforces strict filename requirements that are not immediately obvious to users. Uploads with non-ASCII characters or special symbols fail with cryptic API errors, making it difficult to diagnose why a package upload was rejected. Introduce dedicated validation functions that check filenames against GitLab's allowed character set and verify file accessibility before attempting upload. Early validation with clear error messages helps users identify and fix issues locally rather than discovering them during the upload process. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The package upload workflow requires several file handling capabilities that don't exist yet: calculating checksums for integrity verification, mapping local filenames to different remote names during upload, and collecting files from either explicit paths or directories. Add utility functions that provide these foundational capabilities. The checksum function uses chunked reading for memory efficiency with large files. The file collection function supports two input modes (explicit file list or directory scan) with batch error handling that continues processing valid files when some fail validation. This approach enables callers to report all validation issues at once rather than failing on the first error. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The validators module only handles file validation, but the package upload workflow also requires parsing Git remote URLs and retrieving GitLab tokens. These operations are currently missing, forcing callers to implement their own URL parsing and token resolution logic, which leads to inconsistent error handling and duplicated validation code. Add utility functions for Git URL parsing, GitLab URL normalization, and token retrieval. The URL parsers handle both SSH and HTTPS formats, extracting the GitLab instance URL and project path with consistent error messages. Token retrieval follows a priority chain from CLI argument to environment variable. This centralizes configuration logic in the validators module where related validation already exists, ensuring uniform error handling through ConfigurationError exceptions. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The validators module only handles file and URL validation, leaving configuration validation (Python version, dependencies, tokens, Git setup) scattered or missing entirely. Users encountering setup issues receive generic errors without guidance on resolution, making troubleshooting difficult and increasing support burden. Add a suite of validation functions that check each configuration aspect with detailed, actionable error messages. Each validator follows a consistent pattern: detect the specific issue, explain what failed, and provide platform-appropriate resolution steps. The orchestrating validate_configuration function runs all checks in sequence with appropriate handling for optional Git requirements, allowing callers to validate their entire setup with a single call. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The package upload functionality currently lacks robust error handling and duplicate detection. Network interruptions, rate limits, or server errors cause immediate failures, requiring manual intervention and restart. Additionally, re-uploading files without duplicate awareness leads to wasted bandwidth and potential registry inconsistencies. Introduce an upload orchestration module using tenacity for automatic retry with exponential backoff. The retry logic distinguishes between transient errors (timeouts, rate limits, server errors) and permanent failures (authentication, validation) to avoid futile retry attempts. Duplicate detection supports configurable policies (skip, replace, error) to handle existing files gracefully, while checksum validation ensures upload integrity before marking operations complete. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The CLI currently lacks proper handling of different output environments. When output is piped to a file or consumed by another process, rich terminal formatting with ANSI escape codes and Unicode characters corrupts the output, making it unusable for automation pipelines. Introduce a dedicated formatting module that detects terminal capabilities at runtime and adapts output accordingly. The module supports three output modes: rich console output with colors and Unicode for interactive terminals, structured JSON for machine consumption, and plain ASCII text for non-TTY environments. This separation ensures the tool integrates cleanly into CI/CD pipelines while maintaining a good user experience in interactive sessions. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The formatters module lacked test coverage, making it difficult to refactor output formatting logic or verify behavior across different terminal environments. Without tests, changes to terminal detection, color support, or output rendering could silently break user-facing output. Introduce a complete test suite covering terminal detection, rich console output, JSON output, plain text output, error formatting, and progress display. Mock Rich console and spinner dependencies to ensure tests run reliably without real terminal rendering, enabling consistent CI execution regardless of environment. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The previous CLI module was a placeholder that delegated to an external script via runpy. This approach prevented proper package installation and made the tool unusable when installed via pip, as the standalone script path resolution failed outside the development environment. Replace the placeholder implementation with a complete argparse-based CLI that handles all argument parsing, validation, and shell completion natively. This enables standard package distribution while providing comprehensive flag validation with user-friendly error messages. The implementation follows a phased approach, with project resolution and upload orchestration marked as TODO items for subsequent work. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The CLI previously required manual project specification for every invocation, forcing users to provide full project URLs or paths even when running from within a Git repository that already contains this information in its remotes. Introduce GitAutoDetector to discover Git repositories and extract GitLab project information from configured remotes, prioritizing the origin remote when multiple GitLab remotes exist. Add ProjectResolver to validate project paths against the GitLab API and resolve them to project IDs. The auto-detection gracefully skips known non-GitLab hosts like GitHub and Bitbucket, falling back to manual specification with actionable error messages when detection fails. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The CLI module uses ad-hoc print statements for logging and lacks a unified upload context, making it difficult to control output verbosity and pass configuration state to downstream upload components. Introduce RichHandler-based logging that respects verbosity flags and routes output to stderr when JSON mode is enabled, preserving stdout for machine-readable output. Add UploadContextBuilder to construct a fully validated UploadContext containing the GitLab client, configuration, and duplicate detector, which subsequent upload phases will consume. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The CLI previously stopped after resolving the GitLab project context, leaving file collection, upload execution, and result formatting as unimplemented TODO items. Users could not actually upload packages despite the command accepting all necessary arguments. Complete the main function by wiring together file collection, upload execution via the uploader module, and result formatting. Exit codes now consistently derive from exception classes where available, with a centralized mapping table for standard Python exceptions. This ensures predictable exit behavior for scripting and CI integration while keeping error handling logic maintainable. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The gitlab-pkg-upload package lacks unit test coverage for its core data structures and validation logic. Without tests, regressions in model behavior, exception handling, or input validation could go undetected, making the codebase fragile as it evolves. Introduce a complete unit test suite covering the models module (dataclasses, enums, exception hierarchy, and error enhancement functions) and the validators module (file validation, Git URL parsing, token handling, and configuration validation). All tests are isolated using mocks for external dependencies like filesystem, subprocess, and GitPython, ensuring fast and reliable execution without requiring actual GitLab API access or specific system configurations. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The duplicate detection module lacked test coverage, making it risky to refactor or extend its functionality. Without tests, regressions in session-level tracking or remote GitLab API interactions could go unnoticed until runtime failures occur in production uploads. Add a complete unit test suite covering the DuplicateDetector class and its helper functions. The tests validate SHA256 calculation edge cases, retry logic behavior, session registry management, and remote duplicate detection via mocked GitLab API responses. All external dependencies are mocked to ensure test isolation and fast execution. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The uploader module previously lacked test coverage, making it difficult to verify correctness of upload orchestration, retry behavior, duplicate detection, and error handling. Without tests, regressions could go unnoticed during refactoring or feature additions. Introduce a complete unit test suite covering all public functions in the uploader module. The tests validate transient error classification for retry logic, single file upload with dry-run support, checksum validation including case-insensitive comparison and path variations, duplicate handling across all policy modes (skip, replace, error), file deletion from the registry with proper error recovery, and the main upload orchestration including fail-fast behavior. All external dependencies are mocked to ensure test isolation and fast execution. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The CLI module lacked unit test coverage, making it difficult to verify argument parsing, flag validation, Git repository auto-detection, and the main orchestration flow. Without tests, refactoring or extending CLI functionality risks introducing regressions that would only surface during manual testing or production use. Introduce a complete test suite covering all CLI components including verbosity determination, logging setup, argument parsing, flag conflict detection, GitAutoDetector for repository discovery, ProjectResolver for GitLab project resolution, UploadContextBuilder for context creation, and the main function error handling paths. Each test class isolates a specific component with mocked dependencies to ensure fast, deterministic execution. Signed-off-by: Javier Tia <javier.tia@linaro.org>
Currently, integration testing for the GitLab package upload CLI relies on subprocess execution, which introduces overhead, complicates test isolation, and makes debugging harder. Tests spawn separate processes for each CLI invocation, capture output via pipes, and parse results from stdout/stderr. This approach is slower and provides less control over the execution environment. Introduce a comprehensive integration test suite that calls the CLI main() function directly instead of using subprocess. The new test infrastructure captures stdout/stderr via context managers, handles SystemExit exceptions to extract exit codes, and provides structured test helpers for common operations. This approach improves test speed, simplifies debugging, and enables better isolation while maintaining full coverage of CLI functionality including single/multiple file uploads, directory uploads, duplicate handling policies, project resolution, and error scenarios. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The test suite imports utilities from the legacy monolithic gitlab_common module, which has been refactored into the modular gitlab_pkg_upload package. Tests fail to run because the expected import paths no longer exist, and error handling returns generic exceptions rather than typed ones that match the new exception hierarchy. Update all test imports to use the new package structure, pulling CLI utilities from gitlab_pkg_upload.cli and data models from gitlab_pkg_upload.models. Replace bare exception handlers in GitLab API helpers with typed exception handling that distinguishes authentication, project resolution, and network failures. Switch the script executor from subprocess-based execution to direct module invocation, enabling proper exception propagation and eliminating process overhead during test runs. Reuse the validators module for checksum calculation to avoid duplicate implementations. Signed-off-by: Javier Tia <javier.tia@linaro.org>
Integration tests previously performed token validation independently in each test module, leading to duplicated code and inconsistent error messages when the environment was misconfigured. Users running tests without proper setup received cryptic skip messages that did not explain how to resolve the issue. Introduce a session-scoped autouse fixture that validates all environment requirements once before any integration tests execute. This centralizes the validation logic and provides detailed, actionable error messages that guide users through the exact steps needed to configure their environment. The fixture checks for GITLAB_TOKEN, verifies the presence of a Git repository, and confirms GitLab remotes are configured. Signed-off-by: Javier Tia <javier.tia@linaro.org>
Currently, the upload script exists as a standalone script (gitlab-pkg-upload.py) that cannot be easily imported, tested, or distributed. The monolithic structure makes it difficult to maintain, test individual components, and package for distribution. Restructure the codebase into a proper Python package under src/gitlab_pkg_upload/ with modular components (cli.py, models.py, uploader.py, validators.py). Add package metadata in pyproject.toml with entry point for gitlab-pkg-upload command. Update documentation to reflect the new installation model using 'uv pip install -e .' before running tests. Reorganize test structure to separate unit tests (tests/unit/) from integration tests (tests/integration/). Simplify run_tests.py to delegate to 'uv run pytest' instead of managing dependencies directly. Remove obsolete standalone scripts (gitlab-pkg-upload.py, gitlab_common.py) and redundant integration test files. Update test helpers to import from the installed package instead of subprocess execution of scripts. Add uv.lock for reproducible dependency resolution. Add pytest-instafail to test dependencies for immediate failure reporting. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The project currently lacks continuous integration, meaning tests must be run manually on each developer's machine. This makes it easy for regressions to slip through during code review and creates uncertainty about whether code works across different Python versions. Introduce a GitHub Actions workflow that runs the test suite on Python 3.11, 3.12, and 3.13 for every push and pull request. Unit tests run unconditionally while integration tests only execute when GitLab credentials are available, allowing the workflow to function in contexts where secrets are not configured. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The project lacks automated quality checks and a release pipeline. Without CI workflows, code style violations, documentation issues, and broken links can slip into the codebase undetected, and publishing to PyPI requires manual intervention. Introduce three GitHub Actions workflows to automate these concerns. The lint workflow runs Ruff and Mypy on every push and pull request to catch style and type errors early. The docs workflow validates markdown formatting and link integrity in README files. The publish workflow triggers on GitHub releases to build and upload the package to PyPI automatically, removing the need for manual release steps. Signed-off-by: Javier Tia <javier.tia@linaro.org>
Manual code quality enforcement before commits relies on developers remembering to run linting, formatting, and type checking commands individually. This creates inconsistency between local development and CI/CD pipeline results, leading to failed builds and unnecessary review cycles. Introduce pre-commit configuration that automatically runs ruff linting, ruff formatting, and mypy type checking on staged files. This catches issues at commit time rather than in CI, reducing feedback loops and ensuring code quality standards are enforced consistently across all contributors. Signed-off-by: Javier Tia <javier.tia@linaro.org>
Currently there is no standardized way to manage version numbers across the project. Updating versions requires manual edits to multiple files, which is error-prone and inconsistent with semantic versioning best practices. Integrate bump-my-version as a dev dependency with configuration to automatically synchronize version updates between pyproject.toml and the package init file. This approach ensures single-command version bumps that create git commits and tags, following the standard release workflow pattern used by modern Python projects. Signed-off-by: Javier Tia <javier.tia@linaro.org>
Signed-off-by: Javier Tia <javier.tia@linaro.org>
The test suite previously had no coverage tracking, making it difficult to identify untested code paths and ensure adequate test coverage as the project evolves. Additionally, integration tests could run unexpectedly in CI environments where GitLab credentials were misconfigured, leading to confusing failures. Integrate pytest-cov with dual coverage thresholds: a 90% failure threshold that blocks merges with insufficient coverage, and a 95% warning threshold that surfaces coverage regressions early without blocking development. Require explicit RUN_INTEGRATION_TESTS=1 opt-in for integration tests to prevent accidental execution and make the test environment requirements explicit. This separation ensures unit tests run quickly in all environments while integration tests only execute when deliberately enabled with proper credentials. Signed-off-by: Javier Tia <javier.tia@linaro.org>
Users currently have no convenient way to enable tab completion for glpkg commands. Setting up shell completions manually requires understanding argcomplete internals and knowing the correct paths and configuration for each shell, creating unnecessary friction for new users. Add an --install-completion flag that generates and installs the appropriate completion script for bash or zsh. This leverages argcomplete to generate shell-specific scripts and places them in conventional locations (~/.bash_completion.d/ or ~/.zsh/completion/), then displays the necessary shell configuration commands. The approach follows established patterns used by other CLI tools and supports the two most common shells without requiring global argcomplete activation. Signed-off-by: Javier Tia <javier.tia@linaro.org>
Currently the package can only be distributed through PyPI as a wheel or source distribution. Users who want a single-file executable or who prefer not to use pip must either install from source or build manually, which complicates deployment in restricted environments and CI pipelines. Introduce a build script that generates self-contained .pyz archives using Shiv or PEX. These zipapp binaries bundle the package and all dependencies into a single executable file that runs on any system with a compatible Python interpreter. The CI workflow now builds and uploads this artifact alongside standard distributions during releases, giving users a portable installation option that requires no package manager. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The documentation CI workflow only validated README.md and tests/README.md, leaving newly added documentation files unchecked. This creates a gap where broken links or markdown issues in contribution guides, release procedures, and shell completion documentation would go undetected until users encounter them. Extend the markdown linting and link checking steps to include CONTRIBUTING.md, docs/RELEASING.md, and docs/SHELL_COMPLETION.md. This ensures all user-facing documentation maintains consistent quality and link validity through automated verification. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The original package name "glpkg" may conflict with existing packages or reserved names on PyPI, preventing publication to the public registry. A unique, descriptive name ensures the package can be published and discovered by users searching for GitLab package registry tools. Rename the distribution package to "glpkg-cli" while preserving the "glpkg" command name for end users. This follows the common convention of using a "-cli" suffix for command-line tool distributions, avoiding namespace conflicts while maintaining a clean user experience. The documentation is updated to reflect the new installation commands and includes additional guidance for manual releases and binary installation. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The project lacks centralized documentation for its CI/CD pipeline, requiring developers to read through individual workflow YAML files to understand the build and deployment process. This makes onboarding harder and increases the risk of misconfiguring secrets or triggering workflows incorrectly. Add comprehensive workflow documentation covering all four GitHub Actions workflows (test, lint, publish, docs), including their triggers, required secrets, debugging tips, and future improvement recommendations. Include status badges in the README to provide at-a-glance pipeline health visibility, and add the new documentation file to the docs workflow link checker for validation. Signed-off-by: Javier Tia <javier.tia@linaro.org>
Long lines in documentation files make diffs harder to review and conflict resolution more tedious when multiple contributors edit the same sections. Markdown files also display inconsistently across different editors and terminals when lines exceed standard widths. Apply consistent line wrapping at approximately 72 characters across all markdown documentation. This follows the conventional commit message and prose formatting standard, improving readability in terminals, simplifying three-way merges, and ensuring predictable rendering across tools. Tables are reformatted for column alignment and code blocks are adjusted to fit within the standard width. Signed-off-by: Javier Tia <javier.tia@linaro.org>
Static analysis tools report multiple type safety issues throughout the codebase, including untyped function signatures, missing generic type parameters, and improperly typed callables. Additionally, several strings exceed the recommended line length limits, making the code harder to read and causing linter warnings. Add explicit type annotations to function parameters and return types, particularly for generic containers and callable signatures. This enables mypy to perform proper type checking and catch potential bugs at development time rather than runtime. Reformat long strings by splitting them across multiple lines or extracting variables to improve readability while maintaining the same runtime behavior. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The existing test suite lacks coverage for several important code paths including error handling in execute_upload, edge cases in terminal detection, duplicate handling scenarios, and transient error classification. This gaps make it difficult to verify correct behavior during refactoring and could allow regressions to slip through. Extend the test suite with targeted tests for previously uncovered functionality. The execute_upload tests validate the complete flow from project resolution through file upload, including all exception types mapped in EXCEPTION_EXIT_CODE_MAP. The formatter tests cover ConEmu color detection on Windows and duplicate metadata display in both rich and plain output modes. The uploader tests verify GitlabError response code classification for transient vs permanent errors and the fail_fast behavior with remote duplicates. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The README workflow status badges reference placeholder repository paths that do not correspond to an actual GitHub repository. This causes broken badge images and prevents users from seeing the real CI status at a glance. Point all badge URLs to the actual jetm/glpkg repository where the workflows are defined and executed. This ensures badge images resolve correctly and accurately reflect the current build status. Signed-off-by: Javier Tia <javier.tia@linaro.org>
The workflows currently use pip's built-in caching through setup-python, but dependencies are installed via uv. This mismatch means the cache never benefits actual installs since uv maintains its own separate cache directory, resulting in unnecessary network fetches on every CI run. Enable uv's native caching mechanism in setup-uv instead. This ensures the cache aligns with the package manager actually performing installs, reducing CI execution time and network overhead. Signed-off-by: Javier Tia <javier.tia@linaro.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.