Making concurrent go-routines execute parallely

Technical Discussion
#1

Hi,
I’m trying to learn concurrent programming in Go by writing a key-value store similar to LevelDB. In that, I got stuck with a problem handling multiple go-routines.

I have some data on the memory (RAM) and I have to flush it to disk. To do it asynchronously, I started a go-routine for writing the data to disk so that main go-routine can still do other jobs like processing read and write queries. But as I executed, I observed that, the newly spawned go-routine started executing and stopped at the file write line and switched back to main go-routine. As per my understanding, I think file write is a blocking statement and that’s the reason behind this switch between go-routines. But I’m running it on a 12-core local machine so I expect both go-routines to run parallely. Is there something that I’m missing? Inorder to make it work I had to resort to using sync.WaitGroup to actually make the main go-routine wait until this file write finishes. It would be great if someone can tell if its possible to make these 2 go-routines run parallely.

I’m relatively very new to writing concurrent programs and Go is the first language that I started off with in writing concurrent programs.
Any help would be appreciated. Thanks!

#2

Hi @Abeshek ,

I observed that, the newly spawned go-routine started executing and stopped at the file write line and switched back to main go-routine.

Do you observe that the main goroutine stops after spawning the other one?

#3

I’m not completely sure but when I looked into the logs i observed that logs after new go-routine getting spawned were all from the spawned go-routine and not the main go-routine. And then it stops (spawned go-routine) just before file write and then switches back to main go-routine.

#4

Is it always better to use wait groups while dealing with go-routines? I was able to come up with the idea of using wait groups by assuming running it on a single core machine

#5

Try adding a tiny time.Sleep interval to both goroutines. Perhaps the spawned goroutine runs too fast and pours out all its log statements before the main goroutine has a chance to write its own log output in-between.

#6

Yes. Because when the main goroutine exits (that is, when func main ends), then all goroutines are stopped. To avoid this, the main goroutine must wait for the others to complete.

#7

Yes, the spawned go-routines runs too quickly but only until a certain point (where file read is happening). There it switches from spawned to main go-routine.

#8

I guess maybe its happening parallely but its just that we cant infer about that much from just looking at the logs.

#9

Add a time.Sleep(...) call to both goroutines. I bet you will then see the logs coming from both goroutines concurrently.

1 Like
#10

But isn’t that just go-routines switching back and forth due to block statements in each of their context?
Assuming time.sleep is blocking here

#11

No, the scheduler is pre-emptive, at least partially. It used to be cooperative in some older version of Go but that changed.

This playground snippet shows how two goroutines behave with and without a sleep delay. Run it a few times with delay = 0 and you’ll see that the runtime switches between goroutines. (With delay = 1ns, the switches are of course obvious.)

1 Like
#12

That’s really cool! Thanks for helping me out. Much appreciated!

#13

Oh and this code sample even has no blocking I/O like Print calls or similar.

Run this locally (not sure if the Playground provides more than one core tbh) and you can see a few switches between the goroutines. (On my machine, I have to run the code multiple times to see the switching. Most of the time, it really looks as if both goroutines run one after another.)