[reverse_proxy] No possibility to force a NO_PROXY for reverse_proxy directive

[rse173]

Hello,

I have a use case where I'm behind a proxy, the HTTP(S)_PROXY environment variables are set, and my reverse_proxy directive uses placeholders (i.e. Caddy doesn't know in advance the name of the hostname). All the webservers are in a local network (docker network).

It looks like this :

reverse_proxy {http.request.host.labels.2}-webserver:8080

It worked fine until v2.7.0 when the ProxyFromEnvironment was added to the reverse_proxy directives.
Now Caddy tries to use the proxy if I don't add the webserver hostname in the NO_PROXY environment variable, which I can't because I don't know new webservers that'll come (we also have tls on demand activated).

I found a solution (in a fork) by adding the possibility to add "no_proxy" to the forward_proxy_url setting, which set the proxy to nil. The reverse proxy directive looks like this using this setting :

reverse_proxy {http.request.host.labels.2}-webserver:8080 {
    transport http {
        forward_proxy_url "no_proxy"
    }
}

Is there a better way I didn't find ?

Thanks

[mohammed90]

I don't understand. We use ProxyFromEnvironment if the ForwardProxyURL is not set:

caddy/modules/caddyhttp/reverseproxy/httptransport.go

Lines 330 to 339 in 4ade967

	if h.ForwardProxyURL != "" {
	pUrl, err := url.Parse(h.ForwardProxyURL)
	if err != nil {
	return nil, fmt.Errorf("failed to parse transport proxy url: %v", err)
	}
	caddyCtx.Logger().Info("setting transport proxy url", zap.String("url", h.ForwardProxyURL))
	proxy = http.ProxyURL(pUrl)
	} else {
	proxy = http.ProxyFromEnvironment
	}

I apparently have a related draft PR that I've forgotten (#6399), but not 100% if it's helpful or not 🤔 I need to dwell on the issue.

I found a solution (in a fork)

What's this fork you're speaking of?

[rse173]

I don't want to use ProxyFromEnvironment. As I'm behind a proxy and my HTTP(S)_PROXY var are set, caddy will use the proxy to access the webserver, which I don't want, because all my webservers are in a local (docker) network.

I could use NO_PROXY var, but as I don't know future webservers that will connect, I can't add them to the environment variable. I need a way to tell caddy to not use the proxy for that specific reverse_proxy directive.

The post you mention is linked to my problem, because I can't set NO_PROXY=X_webserver,Y_webserver,Z_webserver, etc. Nor NO_PROXY="*-webserver".

Using the ForwardProxyURL allow to change the proxy URL for a specific reverse_proxy, but I would like to be able to do the opposite and not using any proxy for a reverse_proxy.

The fork I mentioned is my own, here is the diff :

diff --git a/modules/caddyhttp/reverseproxy/httptransport.go b/modules/caddyhttp/reverseproxy/httptransport.go
index 80a49806..56b37a65 100644
--- a/modules/caddyhttp/reverseproxy/httptransport.go
+++ b/modules/caddyhttp/reverseproxy/httptransport.go
@@ -298,7 +298,10 @@ func (h *HTTPTransport) NewTransport(caddyCtx caddy.Context) (*http.Transport, e

        // negotiate any HTTP/SOCKS proxy for the HTTP transport
        var proxy func(*http.Request) (*url.URL, error)
-       if h.ForwardProxyURL != "" {
+       if h.ForwardProxyURL == "no_proxy" {
+               caddyCtx.Logger().Warn("setting transport proxy to nil --> no proxy")
+               proxy = nil
+       } else if h.ForwardProxyURL != "" {
                pUrl, err := url.Parse(h.ForwardProxyURL)
                if err != nil {
                        return nil, fmt.Errorf("failed to parse transport proxy url: %v", err)

It's working as expected, I still have my HTTP(S)_PROXY var, my NO_PROXY var, and I can tell caddy to not use any proxy for reverse proxying to a hostname which is not in the NO_PROXY.

I hope it is clear, I may give a full scenario in example if needed.

Thanks

[mohammed90]

I guess my PR could include a proxy provider "from": "none" 🤔

[rse173]

Yes it should be enough I think. 👌