feature: handle cnames, random ip from list when list, better handling of host.docker.local

[gilleyj]

Still need to set up for new version to push to gemfurry

Goals here:

First, the DNS package doesn't like using "names" for nameserver (go figure) so we need to be able to resolve the special case 'host.docker.internal' to an IP and use that in the nameserver list when the override AGENT_URI contains that.

Second, currently consul_srv only worked when consul responded to a SRV request that contained an IP in the "address" field. Since it's part of the spec that consul can also support a host name here, we need to support that.

Third, we need to not change the lookup behavior much because of the non threaded, synchronous nature of python and dns. (this is a known problem and why there is the weird loop with random sleeps, a brute force method to work through the issue)

Some notes:

Behavior when CONSUL service is registered with an IP:
SRV request for an existing service addrservice registered with the ip of 4.3.2.1 returns:

addrservice.service.consul. 0	IN	SRV	1 1 443 04030201.addr.dc1.consul.

04030201.addr.dc1.consul. is an internal consul record that we make a standard query for against consul returning:

04030201.addr.dc1.consul. 0	IN	A	4.3.2.1

so the returned address should be 4.3.2.1

SRV request for an existing service hostservice registered with the hostname of hostservice.mksp.co:

hostservice.service.consul. 0 IN	SRV	1 1 443 hostservice.mksp.co.

hostservice.mksp.co. is an EXTERNAL host name that returns when queried against consul:
NXDOMAIN - host doesn't exist
but when we query against the "normal" dns (no consul overrides) we get:

hostservice.mksp.co.	77	IN	A	52.71.81.212
hostservice.mksp.co.	77	IN	A	34.231.55.73
hostservice.mksp.co.	77	IN	A	35.169.11.217

so the returned address should be one of the three at random

This changes the original behavior a bit which originally depended on using the additional section of the SRV request to get the IP

FULL SRV lookups for reference:
SRV addrservice asked against consul:

;; QUESTION SECTION:
;addrservice.service.consul.	IN	SRV

;; ANSWER SECTION:
addrservice.service.consul. 0	IN	SRV	1 1 443 04030201.addr.dc1.consul.

;; ADDITIONAL SECTION:
04030201.addr.dc1.consul. 0	IN	A	4.3.2.1
172.17.0.5.node.dc1.consul. 0	IN	TXT	"consul-network-segment="

SRV hostservice asked against consul:

;; QUESTION SECTION:
;hostservice.service.consul. IN	SRV

;; ANSWER SECTION:
hostservice.service.consul. 0 IN	SRV	1 1 443 hostservice.mksp.co.

;; ADDITIONAL SECTION:
172.17.0.5.node.dc1.consul. 0	IN	TXT	"consul-network-segment="

RESOLV request asked against not consul:

;; QUESTION SECTION:
;hostservice.mksp.co.		IN	A

;; ANSWER SECTION:
hostservice.mksp.co.	77	IN	A	52.71.81.212
hostservice.mksp.co.	77	IN	A	34.231.55.73
hostservice.mksp.co.	77	IN	A	35.169.11.217

[alexpatz]

I'm wondering when is a returned from lookup_host going to be False? I see you set d=False in the function but as Karl says this d value doesn't then get appended to iplist. You may need to - instead of setting d to False - just add a line at the end of the function along the lines of:
if len(iplist) == 0:
return False
else:
return iplist[random.randint(0,len(iplist)-1)]

[alexpatz]

You may want to catch DNSException here? Like this example. Not sure what you would do if there was an exception though.
https://www.programcreek.com/python/example/97692/dns.resolver.Resolver