Handling multiple ports for a docker container

[alexturek]

Hey Jason,

I think these two projects (docker-register and docker-discover) are filling a really useful niche for my company, and I'd like to add support for multiple TCP ports. I just wanted to walk through the fix to make sure it was a change you agreed with.

Since docker-discover already exposes a service with etcd keys like this:

/backends/<container name>/port = <port exposed by HAProxy>
/backends/<container name>/<container id> = <docker host:exposed docker port>

I was thinking we could just suffix with the port used by the process-in-the-container. E.g. if you have a docker ps like

CONTAINER ID   IMAGE           ....   PORTS                                             NAMES
xxxyyyzzz      some-web-process       0.0.0.0:49153->8000/tcp, 0.0.0.0:49154->9000/tcp  myservice

You could expose it in etcd with these keys:

/backends/myservice-8000/port = "80"
/backends/myservice-8000/xxxyyyzzz = "my.ip.he.re:49153"
/backends/myservice-9000/port = "5000"
/backends/myservice-9000/xxxyyyzzz = "my.ip.he.re:49154"

effectively treating a service with two ports like two separate services from the HAProxy perspective. This wouldn't require changing docker-discover at all.

[alexturek]

I just realized.. I'm probably going to have to change docker-gen for this as well, right?

EDIT:
Scratch that, I just have to change the template

[alexturek]

Turns out I did need to make a change to docker-gen.

nginx-proxy/docker-gen#29 adds a template function to convert e.g.

quay.io/yourcompany/dockerimage

to

quay.io-yourcompany-dockerimage"

[jwilder]

Thanks for this! This seems like a reasonable approach. I need to test it out a bit first though.

[alexturek]

Alright. I'm totally new to HAProxy myself, so I'm not sure if '-'s are a good character to use in the config, or even what the ideal config looks like

[RuslanZavacky]

@alexturek @jwilder is there any news, or maybe there is better approach to this task described in http://jasonwilder.com/blog/2014/07/15/docker-service-discovery/ ?

[alexturek]

@RuslanZavacky I've since moved on from the company where I was working on this, but here's roughly what we did:

On our CoreOS fleet, we left docker-discovery and docker-register (my fork supporting multiple TCP ports) running at all times. We just made a global list of ports (we only had a half dozen services or so) and we reserved each port for a specific service/port tuple. I.e. FooService:8080 is always exposed on port 7300 locally.

We then configured our services (via docker environment variable config) to talk to a specific IP:port combo.

In other words we did almost exactly what Jason had in his blog post.

If I were doing this again from scratch I'd probably start with Vulcan. It's a higher level proxy that is HTTP based. For TCP though, your best bet is probably running something similar to above.