Find an alternative to bind mounts for STI

[ncdc]

STI creates a temporary working directory and places custom scripts, default scripts, source code, and saved artifacts in it. It uses bind mounts to bind each of these directories into the build container. Bind mounts are fragile when trying to support both host-docker and docker-in-docker builds. We should try to find an alternative to bind mounts.

1 possible option is to have STI encode everything that is currently bind mounted and stream it into the build container. A possible encoding could be tar. The build container would have to support being able to decode the stream, meaning all builder images would need to have the necessary tooling installed to do the decoding.

@bparees @smarterclayton @pmorie

[bparees]

Are we committed to supporting both DIND and host-docker? I sort of assumed we'd reach a point where one clearly won out.

[ncdc]

We don't have to do anything right now to support DIND. @smarterclayton has asked us to focus on host-docker but (afaik) still support DIND. Right now host-docker doesn't work, and if we've got to do something to fix this, it would be nice to avoid bind mounts. If we go with some sort of streaming approach, we'll need to add streaming support for builds (and @pmorie and @ironcladlou and I have discussed a couple of ways to do that).

[bparees]

@ncdc can you elaborate on what is "fragile" about them? The security+unique naming issues on the host directory? or something else?

[ncdc]

@bparees a few things:

1. We have to create a temp dir on the host, which means something needs to clean it up
2. The full path to the temp dir on the host must match the full path to the temp dir in the STI container, since STI will do a bind mount from /tmp/whatever in what it thinks is the STI container, but in reality is the host (for host-docker).
3. SELinux denials/issues
4. File ownership issues

[bparees]

ok, I know @mfojtik had done a lot of digging on tar/streaming into containers, we should get his thoughts/experiences.

[ncdc]

Food for thought - we could use 1 container in a pod to accept the stream (a small image that is capable of decoding tar or whatever) and place the decoded contents into a shared volume. The STI container would also use the shared volume, polling for the existence of some "ready" marker file in the shared volume. Once it sees it, it could do its thing. This way, builder images don't have to have tar installed or know how to decode the stream.

[smarterclayton]

Sounds good, try it.

On Sep 17, 2014, at 5:06 PM, Andy Goldstein notifications@github.com wrote:

Food for thought - we could use 1 container in a pod to accept the stream (a small image that is capable of decoding tar or whatever) and place the decoded contents into a shared volume. The STI container would also use the shared volume, polling for the existing of some "ready" marker file in the shared volume. Once it sees it, it could do its thing. This way, builder images don't have to have tar installed or know how to decode the stream.

—
Reply to this email directly or view it on GitHub.

[VojtechVitek]

@bparees it was me :)

So what I did.. I used busybox image with its built-in tar cmd.

1. Create Data volume first:
   docker run -v /data --name data busybox true
2. Copy whole directory (and its files) using tar -> tar pipe into the Data volume:
   tar -cf - $STI_DIR | docker run --volumes-from data -w /data -i -a STDIN busybox tar -xvf -
3. Mount the Data volume (with the copied STI files) in the Builder itself:
   docker run --volumes-from data <BUILDER-IMAGE-USING-DATA-DIR>

[mfojtik]

@VojtechVitek @bparees I did the self-contained builder, where I streamed artifacts from previous build to a new build (ran restore-artifacts and check if the input is a stream, save-artifacts just returned the data into stdout)

[VojtechVitek]

I randomly came across this project made by the Docker employee last Friday:
https://github.com/discordianfish/docker-backup

..and seems like he's doing basically the same what I did (streaming tar data from/to containers) but completely in Go, using "archive/tar" Writer/Reader (https://github.com/discordianfish/docker-backup/blob/master/backup/backup.go#L51).

[VojtechVitek]

fyi @csrwng