HTTPS + Multiple Hosts + Autocert

I am here because I am struggling to find examples of how you can use autocert for multiple hosts with net/http. I am NOT looking for a copy & paste solution, I genuinely want to see & explore methods of how this can be achieved.
The NGINX reverse proxy with certbot is not a viable solution to my requirements. I need to do this with net/http + autocert or a similar ACME2 autocert equivalent.
Why the NGINX is not a solution:
I have a need to process request logging in ‘real time’ and without disrupting the flow of the request. Using middleware via Alice I spawn a go routine to process the request logging separately (passed to a seperate microservices service) from the normal http flow. This is why the NGINX solution does not work.
I have spent weeks looking at such as chi, echo (as a new user can not add further links) etc. documentation, examples and discussions appear to be targeted towards API development and while I will be using chi within other services to handle any API requirements, finding a net/http + multi hosts using autocert solution is proving difficult.

The multiple hosts are served as static content via http.FileServer, any API paths being redirected off to other services.

Such as caddy etc loose in terms of performance over net/http so are not viable solutions.

I have no issue creating a HTTP solution, but upgrading to a HTTPS solution the ‘multi host’ requirement I have hit a brick wall, hence my call for help from the go community.
I am relatively new to go (~ 1years experience) although I have worked as a software developer for 20+ years.
Any, pointers, links or suggestions would be very much appreciated.

I’m not one hundred percent sure that I understand your use case, but have you seen this?: https://brendanr.net/blog/go-docker-https/

Hi Curtis,
While I appreciate your reply, like most similar examples that I have seen, Example, it does not deal with http -> https redirection for multiple hosts.
For those that may be following his thread here are some code extracts & structure comments of what I have so far:
Each host’s root/public_html directory sits within a sites directory: sites -> siteA, sites -> siteB etc. This is where the static content for each host is served from.

I create my router/mux: package router

package router

type dmux struct {
    StaticRoutes map[string]http.Handler
    APIRoutes    map[string]http.HandlerFunc
}

var Router = dmux{
    StaticRoutes: make(map[string]http.Handler),
    APIRoutes:    make(map[string]http.HandlerFunc),
}

LoadStaticSites method on the dmux struct:

func (mx dmux) LoadStaticSites() error {

	fileServer := http.FileServer(http.Dir(path.Join("/var/www/sites", "siteA")))
	mx.StaticRoutes["siteA"] = fileServer
	
	fileServer = http.FileServer(http.Dir(path.Join("/var/www/sites", "siteB")))
	mx.StaticRoutes["siteB"] = fileServer

 return nil
}

The ServeHTTP method of the route/mux is:

func (mx dmux) ServeHTTP(w http.ResponseWriter, r *http.Request) {

if staticHandler := mx.StaticRoutes[r.Host]; staticHandler != nil {

	// This check is done here so that every hosted site will have access to the API handlers
	if APIHandler := mx.APIRoutes[r.URL.Path]; APIHandler != nil {
		// Check for an API Route
		APIHandler.ServeHTTP(w, r)
	} else {
		staticHandler.ServeHTTP(w, r)
	}

} else {
	// Handle host names for which no handler is registered
	http.Error(w, "Forbidden", 403) // Or Redirect?
}

}`

From main.go : package main

func init() {

    err := router.Router.LoadStaticSites()
    if err != nil {
	    fmt.Println("Failed to load sites.")
	    panic(err)
    }

    // Load API routes
    router.Router.APIRoutes[`/email`] = apiHandlers.Email
    router.Router.APIRoutes[`/samples`] = apiHandlers.Samples

}

func main() {

var m *autocert.Manager

var httpsSrv *http.Server

prod := true

log.Printf("PROD: %v",prod  )

if prod {
	m = &autocert.Manager{
		Prompt:     autocert.AcceptTOS,
		HostPolicy: autocert.HostWhitelist("site1.dab", "www.site1.dab", "site2.dab", "www.site2.dab"),
		Cache:      autocert.DirCache("./certs-cache"),
		// Use Letsencrypts staging server for testing
		Client: &acme.Client{
			DirectoryURL: "https://acme-staging-v02.api.letsencrypt.org/directory",
		},
	}

	httpsSrv = makeHTTPServer()
	httpsSrv.Addr = ":443"
	httpsSrv.Handler = router.Router
	httpsSrv.TLSConfig = &tls.Config{GetCertificate: m.GetCertificate}

	go func() {
		fmt.Printf("Starting HTTPS server on %s\n", httpsSrv.Addr)
		err := httpsSrv.ListenAndServeTLS("", "")
		if err != nil {
			log.Fatalf("HTTP TLS Server failed with %s", err)
		}
	}()
}

var httpSrv *http.Server

if prod {

	httpSrv = makeHTTPToHTTPSRedirectServer()
} else {
	log.Println("Running HTTP")
	httpSrv = makeHTTPServer()

	httpSrv.Handler = router.Router
}

//allow autocert handle Let's Encrypt callbacks over http
if m != nil {
	httpSrv.Handler = m.HTTPHandler(httpSrv.Handler)
}

httpSrv.Addr = ":80"

log.Fatal(httpSrv.ListenAndServe())
} // End of main

func makeHTTPServer() *http.Server {
   return &http.Server{
	ReadTimeout:  5 * time.Second,
	WriteTimeout: 5 * time.Second,
	IdleTimeout:  120 * time.Second,
   }
}

This function is where I get all types of issues E.G. circular redirects … when I attempt to run HTTPS. HTTP works OK

func makeHTTPToHTTPSRedirectServer() *http.Server {
    handleRedirect := func(w http.ResponseWriter, r *http.Request) {
	        newURI := "https://" + r.Host + r.URL.String()
	        http.Redirect(w, r, newURI, http.StatusMovedPermanently)
        }
    router.Router.APIRoutes[`/`] = handleRedirect
    router.Router.StaticRoutes[`/`] = http.HandlerFunc(handleRedirect)
    return makeHTTPServer()
}

As I wrote in the original post, I am relatively new to GO with about a years experience, but this is my first encounter with a ‘public facing’ HTTP/HTTPS server.

Most of what I wrote above turns out to be wrong in terms of my understanding, I will leave my previous examples as they may still help some in understanding this issue.

In essence the ‘multi host’ part of my issue was the root of my misunderstanding:
func main() {

var m *autocert.Manager

var httpsSrv *http.Server

// Set up middleware
reqProcRoute := alice.New(middleware.VMonitor).Then(router.Router)

prod := true

log.Printf("PROD: %v",prod  )

if prod {
	m = &autocert.Manager{
		Prompt:     autocert.AcceptTOS,
		HostPolicy: autocert.HostWhitelist("site1.dab", "www.site1.dab", "site2.dab", "www.site2.dab"),
		Cache:      autocert.DirCache("./certs-cache"),
		// How to use Letsencrypts staging server for testing
		Client: &acme.Client{
			DirectoryURL: "https://acme-staging-v02.api.letsencrypt.org/directory",
		},
	}

	httpsSrv = makeHTTPServer()
	httpsSrv.Addr = ":443"
	httpsSrv.Handler = reqProcRoute
	httpsSrv.TLSConfig = &tls.Config{GetCertificate: m.GetCertificate}

	go func() {
		fmt.Printf("Starting HTTPS server on %s\n", httpsSrv.Addr)
		err := httpsSrv.ListenAndServeTLS("", "")
		if err != nil {
			log.Fatalf("HTTP TLS Server failed with %s", err)
		}
	}()
}

// Incoming requests on :80 will be handled here
var httpSrv = &http.Server{
	ReadTimeout:  5 * time.Second,
	WriteTimeout: 5 * time.Second,
	IdleTimeout:  120 * time.Second,
	Addr: ":80",
	Handler: reqProcRoute,
}

handleRedirect := func(w http.ResponseWriter, r *http.Request) {
	newURI := "https://" + r.Host + r.URL.String()
	http.Redirect(w, r, newURI, http.StatusMovedPermanently)
}

//If Autocert.Manager != nil then handle letsencrypt :80 traffic
if m != nil {
	httpSrv.Handler = m.HTTPHandler(httpSrv.Handler)
} else {
	// Redirect all HTTP -> HTTPS. TODO : This will not work for Letsencrypt challenges
	httpSrv.Handler = http.HandlerFunc(handleRedirect)
}

log.Fatal(httpSrv.ListenAndServe())

} // End of main.

Take note of the redirectHandler, it will parse the ‘host’ value of any request regardless of how many hosts there are, so a single redirect handler for ALL HOSTS will work. Where I had believed that ‘each host’ would need to be handled separately!

Now that the ‘multi host’ issue has been removed I am now left with what has always been my core issue:

How can I redirect all HTTP -> HTTPS and still allow Letsencrypt access via HTTP for its HTTP challenges?