Hello,
I am trying to send multiple requests using http2 package while I got multiple tcp connection once I use this line of code:
conntransport := &http2.Transport{
DialTLS: func(network, addr string, cfg *tls.Config) (net.Conn, error) {
return net.Dial(network, addr)
},
AllowHTTP: true,
}
client := &http.Client{
Transport: conntransport,
}
http.NewRequest(…)
client.Do()
or
client.Post()
Your help is much appreciated
Are you talking about http2’s multiplexing? If so take a look at this thread in golang nuts. It might have some useful examples for you to follow. Also, I don’t see in your code where you are trying to send multiple requests.
Hi @Dean_Davidson,
Yes, I am trying to applu http/2 multiplexing, this is my code and I received multiple tcp connection:
package main
import (
“crypto/tls”
“fmt”
“io/ioutil”
“log”
“net”
“net/http”
“net/http/httptest”
“os”
“time”
“golang.org/x/net/http2”
)
func main() {
stallResp := make(chan bool)
cst := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Printf("Request proto: %s\n", r.Proto)
<-stallResp
w.Write([]byte("Done here"))
}))
if err := http2.ConfigureServer(cst.Config, new(http2.Server)); err != nil {
log.Fatalf("Failed to configure HTTP/2 server: %v", err)
}
cst.TLS = cst.Config.TLSConfig
cst.StartTLS()
tr := &http2.Transport{
TLSClientConfig: cst.Config.TLSConfig,
AllowHTTP: true,
DialTLS: func(network, addr string, cfg *tls.Config) (net.Conn, error) {
return net.DialTCP(network, clienta, servAddr)
},
}
tr.TLSClientConfig.InsecureSkipVerify = true
client := &http.Client{
Timeout: 400 * time.Millisecond,
Transport: tr,
}
url := "https://example:port/"
for i := 1; i <= 50; i++ {
res, err := client.Get(url)
if err == nil {
res.Body.Close()
}
}
defer cst.Close()
close(stallResp)
}
Yeah that sounds exactly like what was happening in the thread I linked to. Specifically this reply which solved the problem:
On Thursday, August 3, 2017 at 11:07:50 AM UTC+2, pala.d…@gmail.com wrote:
Example code with DuckDuckGo as default target:This looks like a thundering herd race on connection setup. So there is no connection to reuse as none of them has been setup yet.
Try completing a single request and then run the rest of the requests in parallel afterwards.
So, take a look at the go playground example above, modify your code to complete a single request first and then run the rest of them in parallel. Also, I don’t think you need to be using golang.org/x/net/http2 directly. Per the docs:
This package is low-level and intended to be used directly by very few people. Most users will use it indirectly through the automatic use by the net/http package (from Go 1.6 and later). For use in earlier Go versions see ConfigureServer. (Transport support requires Go 1.6 or later)
You can probably use net/http, which includes transparent support for http2 as long as you’re using go 1.6 or later (and I’m assuming you are because that is from something like 2015):
Starting with Go 1.6, the http package has transparent support for the HTTP/2 protocol when using HTTPS. Programs that must disable HTTP/2 can do so by setting Transport.TLSNextProto (for clients) or Server.TLSNextProto (for servers) to a non-nil, empty map. Alternatively, the following GODEBUG environment variables are currently supported:
GODEBUG=http2client=0 # disable HTTP/2 client support GODEBUG=http2server=0 # disable HTTP/2 server support GODEBUG=http2debug=1 # enable verbose HTTP/2 debug logs GODEBUG=http2debug=2 # ... even more verbose, with frame dumps
I tried to send one request before doing the loop and I got multiple tcp connection.
Here is my code
package main
import (
"crypto/tls"
"fmt"
"log"
"net"
"net/http"
"net/http/httptest"
"time"
"golang.org/x/net/http2"
)
func main() {
stallResp := make(chan bool)
cst := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Printf("Request proto: %s\n", r.Proto)
<-stallResp
w.Write([]byte("Done here"))
}))
if err := http2.ConfigureServer(cst.Config, new(http2.Server)); err != nil {
log.Fatalf("Failed to configure HTTP/2 server: %v", err)
}
cst.TLS = cst.Config.TLSConfig
cst.StartTLS()
tr := &http2.Transport{
TLSClientConfig: cst.Config.TLSConfig,
AllowHTTP: true,
DialTLS: func(network, addr string, cfg *tls.Config) (net.Conn, error) {
return net.Dial(network, addr)
},
}
tr.TLSClientConfig.InsecureSkipVerify = true
client := &http.Client{
Timeout: 400 * time.Millisecond,
Transport: tr,
}
url := "https://1example:port/"
res, err := client.Get(url)
if err == nil {
res.Body.Close()
}
for i := 1; i <= 50; i++ {
res, err := client.Get(url)
if err == nil {
res.Body.Close()
}
}
defer cst.Close()
close(stallResp)
}
How are you verifying that it’s using multiple connections?
Also, without concurrency/async (your current code is synchronous) my understanding is that the benefit from http2 is probably limited as the whole point of http2’s multiplexing vs http1 pipelining is that you don’t have to wait for things to return concurrently.
Finally, I’m wondering if http tracing might be of interest to you. For example, I used tracing like so:
package main
import (
"fmt"
"log"
"net/http"
"net/http/httptrace"
"sync"
)
// transport is an http.RoundTripper that keeps track of the in-flight
// request and implements hooks to report HTTP tracing events.
type transport struct {
current *http.Request
}
// RoundTrip wraps http.DefaultTransport.RoundTrip to keep track
// of the current request.
func (t *transport) RoundTrip(req *http.Request) (*http.Response, error) {
t.current = req
return http.DefaultTransport.RoundTrip(req)
}
// GotConn prints whether the connection has been used previously
// for the current request.
func (t *transport) GotConn(info httptrace.GotConnInfo) {
fmt.Printf("Connection reused? %v\n", info.Reused)
}
func main() {
tr := &transport{}
client := &http.Client{Transport: tr}
wg := sync.WaitGroup{}
url := "https://www.google.com/"
for i := 1; i <= 5; i++ {
// Increment the WaitGroup counter.
wg.Add(1)
// Launch a goroutine to fetch the URL.
go func(reqNum int) {
fmt.Println("Beginning request #", reqNum)
// Decrement the counter when the goroutine completes.
defer wg.Done()
// Fetch the URL.
req, _ := http.NewRequest("GET", url, nil)
trace := &httptrace.ClientTrace{
GotConn: tr.GotConn,
}
req = req.WithContext(httptrace.WithClientTrace(req.Context(), trace))
if resp, err := client.Do(req); err != nil {
log.Fatal(err)
} else {
fmt.Printf("Got response for #%v using %v\n", reqNum, resp.Proto)
resp.Body.Close()
}
}(i)
}
wg.Wait()
}
… which produced the following output:
$ go run main.go
Beginning request # 5
Beginning request # 1
Beginning request # 2
Beginning request # 3
Beginning request # 4
Connection reused? false
Connection reused? true
Connection reused? true
Connection reused? true
Connection reused? true
Got response for #3 using HTTP/2.0
Got response for #2 using HTTP/2.0
Got response for #5 using HTTP/2.0
Got response for #4 using HTTP/2.0
Got response for #1 using HTTP/2.0
… which is exactly what I wanted. My connection was reused for all subsequent requests.
Yes I got the same when I am requesting from goodle.com.
In case, I am requesting from a server which support http1.1 and http2 , it will choose http1.1 since I did not mentioned in the transport to use http2.
Beginning request # 5
Beginning request # 3
Beginning request # 1
Beginning request # 2
Connection reused? false
Beginning request # 4
Connection reused? false
Connection reused? false
Connection reused? false
Got response for #5 using HTTP/1.1
Got response for #3 using HTTP/1.1
Connection reused? false
Got response for #1 using HTTP/1.1
Got response for #4 using HTTP/1.1
Got response for #2 using HTTP/1.1
If you can help me, how to specify the http2 in a simple way, I will be grateful.
Thank you so much
Well, it should automatically choose http2 in that case. Using the code I showed above, can you point to a public server that supports both http1.1 and http2 and have it select http1.1? I’m wondering if the problem is in whatever server you’re trying to hit not supporting http2.
Only other thing I can think of is to check out net/http/transport.go line 278:
// ForceAttemptHTTP2 controls whether HTTP/2 is enabled when a non-zero
// Dial, DialTLS, or DialContext func or TLSClientConfig is provided.
// By default, use of any those fields conservatively disables HTTP/2.
// To use a custom dialer or TLS config and still attempt HTTP/2
// upgrades, set this to true.
If you weren’t using http2.Transport that could be relevant since you are using a non-zero DialTLS and TLSClientConfig. Other than that, verify the server supports http2.
yes It support http2 and I did the curl command also but by default since both http1.1 and http2 are enabled. It pick http1.1 that’s why I tried http1.transport because I can force http2 but then I will not be able to get multi request in a single TCP connection
Hi @Dean_Davidson, I was checking and the problem is my server support http1.1 upgrade h2c which mean without TLS so do you think I can implement multiplexing with h2c?
Oh. I’m not sure you can accomplish that using the stdlib. It looks like you already found this workaround but perhaps it isn’t working as intended anymore. Maybe you should open an issue on Github and somebody will have a better idea?
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.