install-from-bindep needs PIP_OPTS

[pcunning-rh]

The install-from-bindep script should also incorporate PIP_OPTS="${PIP_OPTS-}" as is already done in the assemble script to allow for custom pip options.

Current workaround is to set your own ARG CONSTRAINTS= in a Containerfile to use that var, which could get overwritten by the builder

[nitzmahone]

I'm curious about what exactly someone's trying to do with this- you mention customizing constraints, but if you're worried about colliding with the existing CONSTRAINTS, this would just move the problem around (and potentially creates a bunch more, depending on what gets passed in). If CONSTRAINTS is already in use, and someone does PIP_OPTS='-c myconstraints.txt', pip is still going to use whichever -c got passed last and silently ignore the other.

[pcunning-rh]

Hello @nitzmahone!

We needed to source some new collections in a recent run of ansible-builder in our network restricted environment. These new collections of course required some external python dependencies. In our environment, we have an Artifactory proxy for public PyPI. We added ARG PIP_OPTS=-i https://artifactory-internal -- trusted-host artifactory-internal to our Containerfile since it appeared that assemble honors this var and passes it to pip.

So the first half of our image build goes swimmingly and pulls Python dependencies from our internal Artifactory proxy, until it approaches the final stages of the Containerfile, where it hits RUN /output/install-from-bindep && rm -rf /output/wheels. There are additional Python dependencies that were needed during this step and the build failed because A) it didn't make a call to our internal Artifactory proxy, and B) the builder couldn't reach out to the public PyPI. We see install-from-bindep is called via pip3 install $CONSTRAINTS --cache-dir=/output/wheels -r /output/requirements.txt with no PIP_OPTS var, hence not attempting to call our internal Artifactory.

To immediately overcome our issues in our build, we created an ARG CONSTRAINTS=-i https://artifactory-internal -- trusted-host artifactory-internal which thankfully was not overwritten by the logic that dynamically creates the CONSTRAINTS file. So during install-from-bindep the values of CONSTRAINTS acted as the required values of PIP_OPTS

To your point, the chances of a collision of CONSTRAINTS is possible, which is why we are asking for a dedicated PIP_OPTS statement in this PR -- to replicate the already established behavior in assemble.

[nitzmahone]

The design behavior that the multi-stage build implies (for better or worse, it's what our downstream builds need, but probably causes more problems than it solves for typical user cases) is that anything custom going on with the Python deps is handled during the Python build stage. That's the reason the Python build/install in assemble is customizable and why the final install is not- if everything's working properly, the output of the Python build stage contains all the wheels necessary to be installed in the final stage to satisfy the requirements listed (which means it shouldn't need to contact a server at all, because all the wheels are already sitting right there).

Short answer (specific to what you're trying to accomplish): if you can get the additional requirements included during the Python build stage, the need for a custom index server in the final stage should go away entirely. Adding shared PIP_OPTS to the final installation script in this manner is likely to break more things than it fixes (and is also still insufficient for a lot of other customization without requiring manual hackery of the generated container build context and/or scripts). We might do some form of this in 2.0, but if we did, they'd be passed through as build args and specific to both the stage and command (eg, PIP_PYBUILD_INSTALL_OPTS, PIP_PYBUILD_CHECK_OPTS, PIP_FINALBUILD_INSTALL_OPTS).