How to implement localhost proxy which injects Proxy-Authorization header to incoming request and sends it to another remote proxy with Go?

[hbollon (Hugo Bollon)]

Hi!
I’m actually building an automation tool based on Selenium with Go called IGopher and I have had a few user requests to implement native proxy support.
However, I am facing an issue with those with authentication…
I can’t send the proxy credentials to Chrome and without them it asks through an alert box for authentication that I can hardly interact with through Selenium (I’m not even sure it’s possible in headless mode) .

So I thought of an intermediary proxy system hosted locally by my program which will add the “Proxy-Authorization” header and transfer the request to the remote proxy.

Something like this: proxy-login-automator

I’m not very familiar with proxies to be honest, but I tried two approach:
The first one using HandlerFunc:

var (
	localServerHost  string
	remoteServerHost string
	remoteServerAuth string
)

// ProxyConfig store all remote proxy configuration
type ProxyConfig struct {
	IP       string `yaml:"ip"`
	Port     int    `yaml:"port"`
	Username string `yaml:"username"`
	Password string `yaml:"password"`
	Enabled  bool   `yaml:"activated"`
}

// LaunchForwardingProxy launch forward server used to inject proxy authentication header
// into outgoing requests
func LaunchForwardingProxy(localPort uint16, remoteProxy ProxyConfig) error {
	localServerHost = fmt.Sprintf("localhost:%d", localPort)
	remoteServerHost = fmt.Sprintf(
		"%s:%d",
		remoteProxy.IP,
		remoteProxy.Port,
	)
	remoteServerAuth = fmt.Sprintf(
		"%s:%s",
		remoteProxy.Username,
		remoteProxy.Password,
	)

	handler := http.HandlerFunc(handleFunc)

	server := &http.Server{
		Addr:           ":8880",
		Handler:        handler,
		ReadTimeout:    10 * time.Second,
		WriteTimeout:   10 * time.Second,
		MaxHeaderBytes: 1 << 20,
	}

	go func() {
		if err := server.ListenAndServe(); err != nil {
			logrus.Fatal(err)
		}
	}()
	logrus.Infof("Port forwarding server up and listening on %s", localServerHost)

	// Setting up signal capturing
	stop := make(chan os.Signal, 1)
	signal.Notify(stop, os.Interrupt)

	// Waiting for SIGINT (pkill -2)
	<-stop

	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
	defer cancel()
	if err := server.Shutdown(ctx); err != nil {
		logrus.Errorf("Forwarder proxy shutdown failed: %v", err)
	}
	logrus.Info("Forwarder proxy stopped")

	return nil
}

func handleFunc(w http.ResponseWriter, r *http.Request) {
	// Inject proxy authentication headers to outgoing request into new Header
	basicAuth := "Basic " + base64.StdEncoding.EncodeToString([]byte(remoteServerAuth))
	r.Header.Add("Proxy-Authorization", basicAuth)

	// Prepare new request for remote proxy
	bodyRemote, err := ioutil.ReadAll(r.Body)
	if err != nil {
		http.Error(w, err.Error(), http.StatusInternalServerError)
		return
	}

	/*Preparation of the new request
		Part I'm not sure about */

	// Create new request
	hostURL := fmt.Sprintf("%s://%s", "http", remoteServerHost)
	proxyReq, err := http.NewRequest(r.Method, hostURL, bytes.NewReader(bodyRemote))
	if err != nil {
		http.Error(w, "Could not create new request", 500)
		return
	}

	// Copy header
	proxyReq.Header = r.Header
	logrus.Info(proxyReq)
	
	/* end of request preparation */

	// Forward request to remote proxy server
	httpClient := http.Client{}
	resp, err := httpClient.Do(proxyReq)
	if err != nil {
		logrus.Info(err)
		http.Error(w, "Could not reach origin server", 500)
		return
	}
	defer resp.Body.Close()

	logrus.Infof("Response: %v", resp)

	// Transfer header from origin server -> client
	for name, values := range resp.Header {
		w.Header()[name] = values
	}
	w.WriteHeader(resp.StatusCode)

	// Transfer response from origin server -> client
	if resp.ContentLength > 0 {
		io.CopyN(w, resp.Body, resp.ContentLength)
	} else if resp.Close {
		// Copy until EOF or some other error occurs
		for {
			if _, err := io.Copy(w, resp.Body); err != nil {
				break
			}
		}
	}
}

With this code snippet, I’m able to intercept request and update the header. However, as it stands, I only make a CONNECT request to my proxy so it’s completly useless:

&{CONNECT http://<PROXY_IP>:3128 HTTP/1.1 1 1 map[Connection:[keep-alive] Proxy-Authorization:[Basic <auth>] Proxy-Connection:[keep-alive] User-Agent:[Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:86.0) Gecko/20100101 Firefox/86.0]] {} 0x71fe20 0 [] false <PROXY_IP>:3128 map[] map[] <nil> map[]   <nil> <nil> <nil> 0xc0000260e0} 

I think I need to change the NewRequest URL but I don’t know what to put there…

And I also try with NewSingleHostReverseProxy:

func PrintResponse(r *http.Response) error {
	logrus.Infof("Response: %+v\n", r)
	return nil
}

// LaunchForwardingProxy launch forward server used to inject proxy authentication header
// into outgoing requests
func LaunchForwardingProxy(localPort uint16, remoteProxy ProxyConfig) error {
	localServerHost = fmt.Sprintf("localhost:%d", localPort)
	remoteServerHost = fmt.Sprintf(
		"http://%s:%d",
		remoteProxy.IP,
		remoteProxy.Port,
	)
	remoteServerAuth = fmt.Sprintf(
		"%s:%s",
		remoteProxy.Username,
		remoteProxy.Password,
	)

	remote, err := url.Parse(remoteServerHost)
	if err != nil {
		panic(err)
	}

	proxy := httputil.NewSingleHostReverseProxy(remote)
	d := func(req *http.Request) {
		logrus.Infof("Pre-Edited request: %+v\n", req)
		// Inject proxy authentication headers to outgoing request into new Header
		basicAuth := "Basic " + base64.StdEncoding.EncodeToString([]byte(remoteServerAuth))
		req.Header.Set("Proxy-Authorization", basicAuth)
		// Change host to the remote proxy
		req.URL = remote
		logrus.Infof("Edited Request: %+v\n", req)
		logrus.Infof("Scheme: %s, Host: %s, Port: %s\n", req.URL.Scheme, req.URL.Host, req.URL.Port())
	}
	proxy.Director = d
	proxy.ModifyResponse = PrintResponse
	http.ListenAndServe(localServerHost, proxy)

	return nil
}

Same here, I successfully intercept the request and edit the header but the CONNECT request failed to resend this time with this error message:

INFO[0007] Pre-Edited request: &{Method:CONNECT URL://google.com:443 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Proxy-Connection:[Keep-Alive] User-Agent:[curl/7.68.0]] Body:<nil> GetBody:<nil> ContentLength:0 TransferEncoding:[] Close:false Host:google.com:443 Form:map[] PostForm:map[] MultipartForm:<nil> Trailer:map[] RemoteAddr:127.0.0.1:46672 RequestURI:google.com:443 TLS:<nil> Cancel:<nil> Response:<nil> ctx:0xc0002986c0}  function=func1 line=58

INFO[0007] Edited Request: &{Method:CONNECT URL://google.com:443 Proto:HTTP/1.1 ProtoMajor:1 ProtoMinor:1 Header:map[Proxy-Authorization:[Basic aWdvcGhlcjpwYXNzd29yZA==] Proxy-Connection:[Keep-Alive] User-Agent:[curl/7.68.0]] Body:<nil> GetBody:<nil> ContentLength:0 TransferEncoding:[] Close:false Host:http://51.178.42.90:3128 Form:map[] PostForm:map[] MultipartForm:<nil> Trailer:map[] RemoteAddr:127.0.0.1:46672 RequestURI:google.com:443 TLS:<nil> Cancel:<nil> Response:<nil> ctx:0xc0002986c0}  function=func1 line=64

2021/03/14 12:10:07 http: proxy error: unsupported protocol scheme ""

If you have an idea to complete what I did or any other method, that will be great!
I think I misunderstand the request struct and the role of URL/Host fields.

You can find all IGopher sources here: GitHub repository (Proxy stuff excluded)