Concurrent Web Requests

Hi everyone,

Excited to have this new community forum to hang out in!

I’m writing a web service in go that will be handling many small-ish, but concurrent requests over HTTP. As expected, as I turn up the number of concurrent requests on my webserver, the average latency for the requests increase (as does the 99% / long tail). Are there any configuration options I should be looking at to help keep latency down?

Here’s a link to a simple example that is just almost completely stdlib code.

I’m benchmarking with apache bench with a variable concurrent request flag value.

ab -n 100000 -c 100


how many concurrent requests are you able to make before things are able to get out of control? You can use a worker/dispatcher model to regulate many requests at once

It would be great if you could post a graph of mean/median/95% response times under various concurrent request conditions.

Also, I’m assuming you’re using go1.5.1, correct?

Thanks for the replies Conrad & Jason. I think these numbers will answer both of your questions. As I mentioned, nothing seems out of the ordinary since you would expect that more I/O => more latency. The same happens with node, for example. Just wondering if there are any options that might provide a win.

The service i’m writing will be “in-line” for requests in a webapp, so every millisecond is valuable :smile:

Also, these numbers are for 1.5.1

ab -n 10000 -c 10 -k

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       1
Processing:     0    0   0.3      0      10
Waiting:        0    0   0.3      0      10
Total:          0    0   0.3      0      10

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      1
  75%      1
  80%      1
  90%      1
  95%      1
  98%      1
  99%      1
 100%     10 (longest request)

100 simultaneous

ab -n 10000 -c 100 -k

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.3      0       5
Processing:     0    4   2.2      4      19
Waiting:        0    4   2.2      4      19
Total:          0    4   2.2      4      19

Percentage of the requests served within a certain time (ms)
  50%      4
  66%      4
  75%      5
  80%      6
  90%      7
  95%      8
  98%      9
  99%     10
 100%     19 (longest request)

1000 simultaneous

ab -n 10000 -c 1000 -k

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   3.0      0      40
Processing:     0   45  11.5     43     126
Waiting:        0   45  11.5     43     126
Total:          0   45  11.4     43     126

Percentage of the requests served within a certain time (ms)
  50%     43
  66%     48
  75%     51
  80%     53
  90%     59
  95%     64
  98%     77
  99%     88
 100%    126 (longest request)

Looking at those numbers, you’ll be better off redesigning the webapp to require less network round-trips that block the user experience. Network latency is going to dwarf the amount of work your server is doing.

Also note that you may be limited by the throughout of your sockets since you’re benchmarking localhost. I recommend using wrk ( over ApacheBench as well since it behaves much more sanely.


In terms of Linux tuning tips for maximizing Go performance, this is a great article:

The alternative would be connecting the webapp to this service over UDP, but I’d rather have the reliability. Fortunately, i’m not currently expected numbers in the 1000s of concurrent requests range so the ~10ms is OK for the time being.

@elithrar Thanks for the link. I’ll take a look!

@jbuberel Thanks for your link as well.

Apache Bench (ab) produces essentially meaningless results.

Try wrk, weighttp, siege, or boom instead.

There’s a long golang-nuts thread about Go’s HTTP performance. The tl;dr is that it’s pretty damn good out of the box.


Haven’t seen those other tools before. Will definitely take a look.

I had stumbled across that thread already, though it’s from 3 years ago so was wondering how outdated it is.

Anyway, sounds like there’s no magic bullet / config option for squeezing out a bit more performance from the go http lib.

Thankfully Go doesn’t require you to dig through 500 different configuration options to tweak performance to something “reasonable” :wink:

Given that, all of the normal tricks apply: limit IO, cache what you can, and where possible avoid expensive network setups. If your Go server is “middleware” between the client and a backend service you may be able to set longer keep-alives between Go and the backend, and/or have them communicate over a Unix socket instead of localhost if they’re on the same machine.

Huh? UDP can’t get rid of the speed of light. I’m talking about eliminating HTTP requests — can you combine multiple results into a single request? do we actually need this data? can we do these requests in the background? — not about reducing the # of packets.

UDP doesn’t get rid of the speed of light, but it would allow the client to “send and forget” such that it doesn’t wait for a response to come back so the latency cost is just the time to send network packets onto the wire.

I’ve been playing around with combining multiple requests into a single request but that actually is ending in more timeouts so maybe the latency due to concurrency i see in benchmarks isn’t what I’m hitting in practice.

Unfortunately there’s no way to do these requests in the background (thanks PHP…) I think all of your suggestions are totally reasonable and implementations I could have gone for but at the end of the day software development is the art of making compromises and they weren’t tradeoffs I wanted to make.

1 Like

might be slightly off topic, but take a look at … it’s a really neat load tester… all in Go for your pleasure :slight_smile:

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.