Gorilla MUX, best way to accept x amount of requests for handler

Technical Discussion
1

I’m using Gorilla mux as my http router. I have a couple of handlers which take care of image processing. For example, someone can send me a file and my service does some resizing, quality checks, etc. This is king of “heavy” for the service and I’m looking for a way to reduce the amount of concurrent requests my mux router accepts. How can I do that?

// so I have a router
r := mux.NewRouter()

// and I have a route
api.Handle("/renditions", rendition.POSTRequest).Methods("POST")

I want mux to accept only x amount of incoming /renditions-requests but I don’t know how to start with this. Can you guys provide me some directions?

2

What response are you expecting when the server has too many requests?

Instead of rate limiting, you might also consider queuing the processing and running it in the background.

I did a quick search and found some rate limiting middleware. I have not tried any of these (or even looked at them closely), but they might work:

More middleware can be found at https://golanglibs.com/category/http-middleware

3

I saw this reddit thread that might also help.

4

To actually limit the number of concurrent requests being processed, create a channel for requests.

Create goroutines to take requests from the channel using the fan out pattern. Set up N goroutines, where N is the maximum number of concurrent requests you want to process. Each goroutine reads from the channel, processes the data, and responds.

When you get an HTTP POST, check the channel is empty. If it is empty, post the request to the channel to handle. If it isn’t, send back 503 Service Unavailable (Overloaded) because you have >N processing tasks running.

This has the advantage that you can easily change the parallelism at run time, and it’s a real limit on concurrent processing, rather than a rate limit, so it will work correctly even if you can’t predict how long requests will take to process.

1 Like
5

Thanks all, for your reactions. Will try to implement this a.s.a.p.

6

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.