Net/http timeouts

[Zyl1]

There are many resources online which inform about the importance of using a http.Client with a finite timeout instead of just calling http.Get() and the likes right away when using the HTTP protocol. However, this timeout – http.Client.Timeout – encapsulates the entire request. The docs say this about it:

The timeout includes connection time, any redirects, and reading the response body. The timer remains running after Get, Head, Post, or Do return and will interrupt reading of the Response.Body.

What this means is that even a timeout value which, intuitively, seems rather long (such as 30 minutes), can terminate a perfectly fine long-running request in the middle of it. Are there any more pragmatic options to control when a connection should time out? (i.e. minimum bitrate, maximum idle time, etc. ?)

[acim (Boban Acimovic)]

You can use context.Context

https://golang.org/pkg/net/http/#Request.WithContext

You can also use fine grained timeouts by using custom http.Transport:

import (
	"net"
	"net/http"
	"time"
)

// NewClient creates customized *http.Client
func NewClient(timeout time.Duration, transport http.RoundTripper) *http.Client {
	return &http.Client{
		Timeout:   timeout,
		Transport: transport,
	}
}

// NewTransport creates customized *http.Transport.
func NewTransport(dialTimeout, tlsHandshakeTimeout time.Duration, maxIdleConnsPerHost int) http.RoundTripper {
	return &http.Transport{
		Proxy: http.ProxyFromEnvironment,
		DialContext: (&net.Dialer{
			Timeout:   dialTimeout,
			KeepAlive: 30 * time.Second,
			DualStack: true,
		}).DialContext,
		MaxIdleConns:          100,
		IdleConnTimeout:       90 * time.Second,
		TLSHandshakeTimeout:   tlsHandshakeTimeout,
		ExpectContinueTimeout: 1 * time.Second,
		MaxIdleConnsPerHost:   maxIdleConnsPerHost,
	}
}

[Zyl1]

I am not sure if I fully understand. If http.DefaultTransport already has timeouts set, then why are people lobbying so much for setting the deadline timeout for the entire request?