Shared pool of workers

Hello all!

My first post in the forum :slight_smile:

I’ve a question I could not find a proper response anywhere.

I know how to build a pool of workers using go routine without problem. My question is related to a shared pool of workers.

Let’s suppose I’ve a webserver where request are coming every time from different users. My idea is to build a shared pool of workers, it means, a pool shared by all the web sessions.

Creating the pool is no problem, I’m just trying to figure out the best way to collect the results back. I mean, making all the workers writing the results in a same channel will be a mess, since I’ll have results from different sessions in a same channel. Should I provide a “private” channel for each session, so the worker will write on this channel?

Also, how’s the wait condition done in a shared pool? All the examples so far consider a single jobs channel, that are closed when no more jobs are necessary. But what about a shared pool? Let’s say I’ve put two jobs in the worker’s channel. How to make the session wait for all the workers to finish processing these requests?

Thanks in advance for any support and sorry for so many questions!

Yes, create a channel the session will receive the result on. The worker will send the single result over this channel and can the close it.

For a server it makes no sense to terminate the workers when the job channel is empty. This can happen at any time. You will have to react to the system being shut down. Only then should the job channel be closed which will cause the workers to terminate.

Thanks for your feedback @lutzhorn :slight_smile:

Keep in mind that goroutines are cheap to create. In many circumstances it doesn’t make sense to manage a pool of goroutines; instead, create one when needed and let it die when done. If the objective is to limit the amount of concurrent work you can use a semaphore of some kind.

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.