Here is my observation I have multiple fqdn configured and will try to reach it one by one but only one will respond http2 response and others will fail. I have configured clientTimeout 2s in http.Client and will try first fqdn which will not be resolved in my app to ip and port so request tried with fqdn and port but http2 request takes 4s second to timeout instead of 2s.. I am unable to understand the logic behind it why clientTimeout is discarded. Please share the details or any reference
Can you please share some code that shows what you have tried?
2 Likes
There are many timeouts involved in an HTTP-Request. For example DNS-query happens first to resolve the host name, this can already take a while to complete. After that first a TLS-Handshake needs to be established (including cipher-negotiation, certificate validation, optional revocation check), following this there could be HTTP-redirects or protocol switches (like establishing a websocket) and then finally the actual http-request starts with writing headers, writing body and reading the reply.
There are various timeouts for all stages of this complex process and it’s not easy to combine the right timeouts for your use-case.
If you just want a “end-to-end” timeout for your whole function call, you should probably use a context.WithTimeout() and pass that to your calls, this will stop after your timeout, no matter in which step the process currently is.
Thanks for the reply
I have setup only clientTimeout using below and sent request.. After using contextTimeout I am able to achieve 2s exit.. still with clientTimeout code should have exited
client := &http.Client{
CheckRedirect: func(req \*http.Request, via \[\]\*http.Request) error {
return http.ErrUseLastResponse // Stops http stack redirect
},
Transport: &http2.Transport{
AllowHTTP: true,
DialTLS: func(network, addr string, cfg \*tls.Config) (net.Conn, error) {
return net.DialTimeout(network, addr, (2000)\*time.Millisecond)
},
PingTimeout: 2000 \* time.Millisecond,
ReadIdleTimeout: 2000 \* time.Millisecond,
},
Timeout: 2000 \* time.Millisecond,
}
postReq, err := http.NewRequest(http.MethodPost, url, bytes.NewBuffer(data))
resp, err := client.Do(postReq)
if err != nil {
if err, ok := err.(net.Error); ok && (err.Timeout() || strings.Contains(err.Error(), cc.CONN_REFUSED)) {
return resp, err, true
}
return resp, err, false
}
with timeout code
timeout := 2 * time.Second // or whatever you need per request
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
req, err := http.NewRequestWithContext(ctx, method, url, body)
if err != nil {
// handle error constructing request
}
resp, err := client.Do(req)
if err != nil {
// context deadline exceeded will land here
if ne, ok := err.(net.Error); ok && ne.Timeout() {
// timed out
}
return resp, err, false
}
You have to understand that timeouts are happening after each other. So the total maximum time a request can take is the sum of individual timeouts. If I set DNS-timeout, connect-timeout, read-timeout each to two seconds, my total timeout is effectively a max of 6 seconds (if all parts take their maximum amount of time).
If for example your DNS-query takes 1 second, the connect takes 1 second and then your client.Read fails with a timeout after 2 seconds, the total time until this timeout occurs is effectively 4 seconds. A context.Context on the other hand is like a stop-watch you start at the beginning and which just expires after exactly 2 seconds, no matter how long the individual steps each take.
What you usually want is a short timeout for establishing the connection (DNS, connect, TLS-Handshake, redirects) and a long timeout to actually read the answer, depending on the size of the answer. If the server is not responding, you want to timeout after 2 seconds, but if you are downloading 1 GB of data, you want the download not to stop after 2 seconds, but give it enough time. This is why there are many different timeouts - each can be set to a sensible value depending on your use case.