Skip to content

[FEAT] Exit code differentiation: Distinguish config errors from compliance failures #61

@Joseph94m

Description

@Joseph94m

Feel free to propose other implementation details.

Is your feature request related to a problem? Please describe.

Today, Plumber uses exit code 0 for success and 1 for everything else. Whether it's a compliance failure (threshold not met) or an actual runtime error (bad config, network failure, missing token). This makes it impossible for CI/CD pipelines to distinguish between "your pipeline is not compliant" and "Plumber itself broke." Users who want to handle these cases differently (for example, soft‑fail on compliance but hard‑fail on errors) cannot do so.

Describe the solution you'd like

Differentiate exit codes:

Exit Code Meaning
0 Analysis passed (compliance ≥ threshold)
1 Compliance failure (compliance < threshold)
2 Runtime error (configuration error, network failure, missing token, etc.)

This allows CI systems to treat compliance failures differently from internal errors.

Implementation Hints

These are just ideas. Feel free to change the implementation.

  • cmd/root.go currently calls os.Exit(1) for any error returned by rootCmd.Execute().
  • In cmd/analyze.go, runAnalyze returns fmt.Errorf(...) for both configuration/runtime errors and the compliance threshold failure (compliance %.1f%% is below threshold %.1f%%).
  • Introduce a custom error type (for example ComplianceError) in cmd/ or a small shared package to represent "compliance below threshold".
  • In runAnalyze, return a ComplianceError instead of a generic error when the only failure is compliance < threshold.
  • In cmd/root.go’s Execute():
    • If the error is a ComplianceErroros.Exit(1).
    • For any other error → os.Exit(2).
  • Update the Exit codes: section in analyzeCmd.Long to document the new behavior.

Files Touched

  • cmd/analyze.go (wrap threshold failures in a ComplianceError type)
  • cmd/root.go (map ComplianceError to exit code 1, all other errors to exit code 2)

Why It's Valuable

Enables CI/CD pipelines to distinguish between "not compliant yet" and "something is broken." Users can, for example, configure GitLab CI with allow_failure: exit_codes: [1] to soft‑fail on compliance while still treating internal errors as hard failures. This is a common pattern in CLI tools and improves Plumber’s ergonomics in automated environments.

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions