Qubership Pipelines CLI Samples

Sample implementations of CLI commands using qubership-pipelines-common-library

Released as executable zipapp artifact and docker image to use inside Declarative Atlas Pipelines

>> Development Guide <<

Use development guide above to:

• Kickstart developing new commands using common library
• When dev-testing new library features

Using resulting artifact

The intended way to distribute CLI as an artifact is zipping it using ZIPAPP

Example of build process is provided in build_pyz.sh

Please note, that you need to unzip resulting '.pyz' artifact before using - this requirement is caused by zipimport, which is used during zipapp.

It disallows ZIP import of dynamic modules (and qubership dependencies already have '_argon2_cffi_bindings', at least)

So an example process of using this CLI might look like this:

...
# qubership_cli_samples.pyz is built by external pipeline and moved to this machine beforehand
unzip -q ./qubership_cli_samples.pyz -d ./qubership_cli_samples
python qubership_cli_samples calc --context_path=context2.yaml
...

Passing params into CLI

Common library provides special decorator @utils_cli that allows you to:

• configure logging of executed commands (by passing --log-level=INFO, for example, and available values are DEBUG, INFO, WARNING, ERROR, CRITICAL)
• pass the context.yaml filepath (using --context_path=./data/context2.yaml)
• or, instead, manually pass your parameters from CLI (using multiple -p flags with key=value syntax, e.g. -p params.operation=multiply)

To use this decorator, you need to add it together with @cli.command("cli_command_name"), and add **kwargs as input parameters to your function, and pass them into ExecutionCommand constructor (as shown in these examples)

Full example of passing multiple params looks like this:

python qubership_cli_samples calc --log-level=DEBUG -p params.param_1=9 -p params.param_2=10 -p params.operation=multiply -p params.result_name=result_divide

You can also pass multi-level nested params using -p flag, dots and double-underscores are supported as separators:

-p params.credentials.system_name.login=user

Samples Docker Image

This repository also provides package with docker image of built executable zipapp with command samples.

It is available via ghcr.io/netcracker/qubership-pipelines-cli-command-samples:latest (or any tagged release version)

While '.pyz' artifact is built on GitHub-provided ubuntu-latest image and is ready-to-use in your GitHub workflows with the same runner, since we can't directly reuse it as a base image for our docker build, we use a different one.

Currently, base image is bitnami/python:3.11.11-debian-12-r0, since it allows us to run our '.pyz' artifact with all its dynamic libraries requirements, without having to perform additional adaptation.

Minification concerns:

By default, zipapp produces uncompressed artifacts - and our dependencies are already of a considerable size.

So the best course of action is to use zip deflate compression - it's tested and it works.

Compression method	Size	Notes
No compression	36.27 MiB
Minification	29.2 MiB	pyminify --in-place - but it potentially leads to unexpected problems with used dependencies
Minification (with literals removal)	19.3 MiB	pyminify --in-place --remove-literal-statements - also removes docstrings and comments - same concerns
ZIPAPP compression	9.55 MiB	Uses zipapp --compress flag, which compress files with the deflate method
Minification + ZIPAPP compression	7.16 MiB	Still has all minification concerns, and difference with compression is negligible

*These builds were performed on Windows, sizes on Linux differ slightly