Dev Environment Inside Docker Container

How We Got Here

This experimental image is the result of a few separate problems I had that—eventually—I realized were not entirely separate.

• Build reproducibility wasn’t great. My environment was essentially a hodge-podge of… well, random stuff really. Eventually I want to move my builds to a CI tool but that’s pie in the sky for the moment.
• Playing around with new tools was a pain. Sometimes my byzantine environment would interfere with the tool; and if the tool configuration was involved I had to decide if setting everything up was worth the risk of having to tear it back down if I didn’t end up caring for it.
• Certain functions within Emacs were slow, annoying to setup, or sometimes even unavailable on Windows. eg Magit is slow in Windows and instantaneous in Linux. Compiling in C or C++ is much easier in Linux. Development in general is much more sane in Linux.

Standardizing my dev environment configuration and putting it into a linux-based container helps to solve all of these problems.

How to Use this Project

Build Scripts

#!/bin/bash
emacs --batch --eval "(require 'org)" --eval '(org-babel-tangle-file "README.org")'

Build Container

Based on this file:

#!/bin/bash
./tangle.sh
./build.sh

Based on Dockerfile

#!/bin/bash
docker build -t dev-img-base .

Run Container

Script

This script uses nine variables:

1. GITNAME - the name with which to sign git commits
2. GITEMAIL - the email with which to do the same
3. DOTSSH - the path to the .ssh folder containing the id_rsa/id_rsa.pub pair to use for authentication
4. SRCHOME - the directory in which project directories will be stored
5. VIRTHOME - the directory that will act as HOME for the container user
6. USERNAME - the name of the user to create inside the container. this script uses the host user along with their uid and gid, so any new files owned by the container user will be owned by the host user as well.
7. USERID - the id of the host user
8. GROUPNAME - the group name of the host user
9. USERGID - the group id of the host user

I haven’t looked into it too deeply, but I believe the way Docker works requires that DOTSSH, SRCHOME, and VIRTHOME are distinct and don’t any one contain another.

Note: if on WSL2, it’s easiest if all shared directories are native Linux directories. Directories under /mnt/c will NOT work in certain circumstances (eg mapping to a .ssh directory in teh container)! Handling ownership and permissions gets all weird (and it’s already weird enough when the host OS is Linux). Note: run.sh requires the USER environment variable to be set and pointing to a valid user Note: run.sh requires git to be installed and configured

#!/bin/bash
DOTSSH="/home/$USER/.ssh:/mnt/.ssh:ro" 
SRCHOME="/home/$USER/.dv/src:/src"
VIRTHOME="/home/$USER/.dv/virt:/home/$USER"

USERNAME="UNAME=$USER"
USERID="UID=$(id -u $USER)"
GROUPNAME="GRPNAME=$(id -gn $USER)"
USERGID="GRPID=$(id -g $USER)"

GITNAME="GITNAME=$(git config user.name)"
GITEMAIL="GITEMAIL=$(git config user.email)"

docker container run -d \
       -p 55111:55111 \
       -p 60001:60001/udp \
       -v $DOTSSH \
       -v $SRCHOME \
       -v $VIRTHOME \
       -e $GITNAME \
       -e $GITEMAIL \
       -e $USERNAME \
       -e $USERID \
       -e $GROUPNAME \
       -e $USERGID \
       dev-img-base:latest

CLI

Note: the user will default to dev.

docker container run -d \
       -p 55111:55111 \
       -p 60001:60001/udp \
       -v <path/to/.ssh>:/mnt/.ssh:ro \
       -v <path/to/srchome>:/src \
       -v <path/to/virthome>:/home/dev \
       -e "GITNAME=<git-name>" \
       -e "GITEMAIL=<git-email>" \
       dev-img-base:latest

Connect to the Container

Via mosh:

#!/bin/bash
mosh --ssh="ssh -p 55111 -l $USER" --port 60001 localhost

Via ssh:

#!/bin/bash
ssh -p 55111 -l $USER localhost

A Word to the Wise

This repo should in no way be confused for best practice. I have no idea what I’m doing.

The Setup

We start by summoning Ubuntu from the void.

FROM ubuntu:22.04

This line forces tzdata to use UTC instead of interactively asking for our locale data during the build: https://stackoverflow.com/questions/44331836/apt-get-install-tzdata-noninteractive

ENV DEBIAN_FRONTEND=noninteractive

Here we setup some environment defaults that will be used later to create our user. These can be overwritten in the docker container run... command to match the user on the host.

ENV UID="1000" 
ENV UNAME="dev" 
ENV GRPID="1000" 
ENV GRPNAME="dev" 
ENV SHELL="/bin/bash" 

Now to install our base packages. A few notes:

• mosh is a nice utility for running an editor on a remote machine: https://linuxhandbook.com/mosh/
• Though I will eventually be building an emacs environment on top of this image, it’s nice to have vi/vim-tiny around for random small edits that sprout up.
• We need ca-certificates in order to accept TLS handshakes (eg when pushing to github).
• We’ll need locales later to setup the environment for mosh.
• dos2unix is necessary until I get this container bootstrapped. Until then I’m writing this on Windows.

RUN apt-get update && \
    apt-get install -y --no-install-recommends \
      openssh-server \
      mosh \
      bash \
      sudo \
      git \
      vim-tiny \
      ca-certificates \
      locales \
      dos2unix 

Copy our sshd configuration to where sshd can find it.

COPY build/sshd_config /etc/ssh/sshd_config

I kept running into a weird bug when trying to run my container on WSL2:

“Missing privilege separation directory: /run/sshd”

Starting and stopping the ssh service seems to fix it 🤷.

RUN service ssh start
RUN service ssh stop

mosh requires the locale to be set to UTF-8 and for some reason I couldn’t manually set LANG/LANGUAGE/LC_ALL by passing them in as -e parameters in the docker container run... command. Thus these next few lines effectively — if not prettily — accomplish this. see: https://unix.stackexchange.com/questions/280796/mosh-server-needs-a-utf-8-native-locale-to-run and: https://stackoverflow.com/questions/28405902/how-to-set-the-locale-inside-a-debian-ubuntu-docker-container and: http://jaredmarkell.com/docker-and-locales/

RUN sed -i '/en_US.UTF-8/s/^# //g' /etc/locale.gen && \
    locale-gen
ENV LANG en_US.UTF-8  
ENV LANGUAGE en_US:en  
ENV LC_ALL en_US.UTF-8     

Now we can expose the ports for sshd and mosh, respectively.

EXPOSE 55111 60001/udp

Our entrypoint script was written in Windows so we ensure the line endings aren’t a problem before executing it.

COPY build/start.sh /usr/local/bin/start.sh
RUN dos2unix /usr/local/bin/start.sh

ENTRYPOINT ["bash", "/usr/local/bin/start.sh"]

Reference Files

sshd_config

I’m lazy and I don’t want to configure sshd by hand. Someone once said Laziness was a virtue; then again, he did invent Perl…

Port 55111

Protocol 2
AllowTcpForwarding yes
ChallengeResponseAuthentication no

Compression yes
GatewayPorts yes
LogLevel VERBOSE
LoginGraceTime 50s
MaxAuthTries 6
MaxStartups 10
PasswordAuthentication no
PermitRootLogin no
PermitUserEnvironment yes
PidFile /var/run/sshd.pid
PrintLastLog yes
PrintMotd no
PubkeyAuthentication yes
StrictModes yes

TCPKeepAlive no
UseDNS yes

PermitTunnel yes

start.sh

I’m thinking this script might make sense as a standalone repo, where it could be cloned in from github in the Dockerfile. I could remove the -D flag for sshd at the end and then this script could be called as part of the image’s ENTRYPOINT script, which could setup any image-specific stuff and then call bash or something.

Disable the root user and setup our own User. From: https://github.com/JAremko/alpine-vim/blob/master/Dockerfile#L22 We do this here instead of in the Dockerfile so that the “builder” and “runner” of the container don’t have to be the same user.

echo "${UNAME}:x:${UID}:${GRPID}:${UNAME},,,:/home/${UNAME}:${SHELL}" >> /etc/passwd \
    && echo "${UNAME}::17032:0:99999:7:::">> /etc/shadow \
    && echo "${UNAME} ALL=(ALL) NOPASSWD: ALL"> "/etc/sudoers.d/${UNAME}" \
    && chmod 0440 "/etc/sudoers.d/${UNAME}" \
    && echo "${GRPNAME}:x:${GRPID}:${UNAME}" >> /etc/group

So this probably isn’t great but I believe it allows us to copy the host’s keys WITHOUT saving them in the image itself (which would be BAD^TM). Instead we will save the keys under $VIRTHOME/.ssh with read/write privileges reserved for the current user. Anyways, I am once again forced to refer you back to A Word to the Wise.

UHOME="/home/$UNAME"
if [[ -d "$UHOME/.ssh" ]]; then
    echo ".ssh already exists. skipping key copy."
else
    echo ".ssh does not exist. copying keys."
    mkdir -p "$UHOME/.ssh" && chmod 0700 "$UHOME/.ssh"

    cp /mnt/.ssh/id_rsa "$UHOME/.ssh/id_rsa"
    chmod 0600 "$UHOME/.ssh/id_rsa"

    cp /mnt/.ssh/id_rsa.pub "$UHOME/.ssh/id_rsa.pub"
    chmod 0600 "$UHOME/.ssh/id_rsa.pub"

    cp /mnt/.ssh/id_rsa.pub "$UHOME/.ssh/authorized_keys"
    chmod 0600 "$UHOME/.ssh/authorized_keys"

    if [[ -e "/mnt/.ssh/authorized_keys" ]]; then
        cat /mnt/.ssh/authorized_keys >> $UHOME/.ssh/authorized_keys
    fi

    chown -hR "$UNAME:$GRPNAME" "$UHOME"

    ssh-keyscan github.com >> "$UHOME/.ssh/known_hosts"
    ssh-keyscan gitlab.com >> "$UHOME/.ssh/known_hosts"
    ssh-keyscan bitbucket.com >> "$UHOME/.ssh/known_hosts"
fi

Lets configure git real quick so it doesn’t yell at us later

#!/bin/bash

git config --global user.email $GITEMAIL
git config --global user.name $GITNAME

Finally! Lets kick everything off!

Note: we pass -D to sshd to ensure it runs in the foreground and blocks the container from exiting.

mosh-server 
/usr/sbin/sshd -De