dynamic a resolvers being bypassed if entry is in /etc/hosts

[WillPlatnick]

Hello,
Caddy v2.8.4 h1:q3pe0wpBj1OcHFZ3n/1nl4V4bxBrYoSoab7rL9BMYNk= installed via homebrew on macOS

I am trying to setup a local reverse proxy that is doing path-based routing on the same domain, and resolving the host to different IP's, depending on the path.

Here is my Caddyfile

{
  debug
  https_port 443
  log {
    level debug
  }
}

my.site.com {
    tls internal
    @fivesight {
        path /mypath*
    }

    reverse_proxy @fivesight {
        dynamic a {
            name my.site.com
	        port 443
	        refresh 0
                resolvers 127.0.0.1
        }
        transport http {
            tls
            resolvers 127.0.0.1
	    tls_insecure_skip_verify
        }
    }

    @other {
        not path /mypath*
    }

    reverse_proxy @other {
        dynamic a {
            name my.site.com
            refresh 0
            port 443
        }
        transport http {
            tls
            tls_insecure_skip_verify
        }
    }
}

I have a DNS server running on 127.0.0.1 that resolves my.site.com to a specific IP. This works properly. 1.1.1.1 isn't the real IP, I just redacted it.

dig @127.0.0.1 my.site.com

; <<>> DiG 9.10.6 <<>> @127.0.0.1 my.site.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 45733
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
;; QUESTION SECTION:
;my.site.com.		IN	A

;; ANSWER SECTION:
my.site.com.	0	IN	A	1.1.1.1

;; Query time: 0 msec
;; SERVER: 127.0.0.1#53(127.0.0.1)
;; WHEN: Wed Jul 31 11:05:59 EDT 2024
;; MSG SIZE  rcvd: 60

I am testing using the following curl curl https://my.site.com/mypath/ --resolve my.site.com:443:127.0.0.1 -v

If I don't have 127.0.0.1 my.site.com in my /etc/hosts - the setup works perfectly

I get a log showing DNS worked as it should:

2024/07/31 15:04:32.128	DEBUG	http.reverse_proxy.upstreams.a	refreshing A upstreams	{"version": "ip", "name": "my.site.com", "port": "443"}
2024/07/31 15:04:32.492	DEBUG	http.reverse_proxy.upstreams.a	discovered A record	{"ip": "1.1.1.1"}

However, once I add 127.0.0.1 my.site.com to my /etc/hosts, Caddy resolves the DNS to 127.0.0.1, rather than using the specified resolver:

2024/07/31 15:07:48.957	DEBUG	http.reverse_proxy.upstreams.a	refreshing A upstreams	{"version": "ip", "name": "my.site.com", "port": "443"}
2024/07/31 15:07:48.957	DEBUG	http.reverse_proxy.upstreams.a	discovered A record	{"ip": "127.0.0.1"}

I further verified it's taking the hosts file entry by setting it to 127.0.0.2 in the hosts file, and I get this in the logs:

2024/07/31 16:12:53.809	DEBUG	http.reverse_proxy.upstreams.a	refreshing A upstreams	{"version": "ip", "name": "my.site.com", "port": "443"}
2024/07/31 16:12:53.810	DEBUG	http.reverse_proxy.upstreams.a	discovered A record	{"ip": "127.0.0.2"}

[mohammed90]

This is an artifact of how the Go resolver works. It first consults with /etc/hosts, then uses the specified upstream resolver if it's not found. I'm trying to think what would be the best course of action for this scenario. Honestly, your setup is odd. Any good reason for that? How could your DNS server know the path anyways? This doesn't appear like a split-horizon setup.

Anyways, I know this feature was developed per a sponsor request. We'll have to check the original requirements and see where to go from there given this report.