Making concurrent go-routines execute parallely

Technical Discussion
1

Hi,
I’m trying to learn concurrent programming in Go by writing a key-value store similar to LevelDB. In that, I got stuck with a problem handling multiple go-routines.

I have some data on the memory (RAM) and I have to flush it to disk. To do it asynchronously, I started a go-routine for writing the data to disk so that main go-routine can still do other jobs like processing read and write queries. But as I executed, I observed that, the newly spawned go-routine started executing and stopped at the file write line and switched back to main go-routine. As per my understanding, I think file write is a blocking statement and that’s the reason behind this switch between go-routines. But I’m running it on a 12-core local machine so I expect both go-routines to run parallely. Is there something that I’m missing? Inorder to make it work I had to resort to using sync.WaitGroup to actually make the main go-routine wait until this file write finishes. It would be great if someone can tell if its possible to make these 2 go-routines run parallely.

I’m relatively very new to writing concurrent programs and Go is the first language that I started off with in writing concurrent programs.
Any help would be appreciated. Thanks!

2

Hi @Abeshek ,

I observed that, the newly spawned go-routine started executing and stopped at the file write line and switched back to main go-routine.

Do you observe that the main goroutine stops after spawning the other one?

3

I’m not completely sure but when I looked into the logs i observed that logs after new go-routine getting spawned were all from the spawned go-routine and not the main go-routine. And then it stops (spawned go-routine) just before file write and then switches back to main go-routine.

4

Is it always better to use wait groups while dealing with go-routines? I was able to come up with the idea of using wait groups by assuming running it on a single core machine

5

Try adding a tiny time.Sleep interval to both goroutines. Perhaps the spawned goroutine runs too fast and pours out all its log statements before the main goroutine has a chance to write its own log output in-between.

6

Yes. Because when the main goroutine exits (that is, when func main ends), then all goroutines are stopped. To avoid this, the main goroutine must wait for the others to complete.

7

Yes, the spawned go-routines runs too quickly but only until a certain point (where file read is happening). There it switches from spawned to main go-routine.

8

I guess maybe its happening parallely but its just that we cant infer about that much from just looking at the logs.

9

Add a time.Sleep(...) call to both goroutines. I bet you will then see the logs coming from both goroutines concurrently.

1 Like
10

But isn’t that just go-routines switching back and forth due to block statements in each of their context?
Assuming time.sleep is blocking here

11

No, the scheduler is pre-emptive, at least partially. It used to be cooperative in some older version of Go but that changed.

This playground snippet shows how two goroutines behave with and without a sleep delay. Run it a few times with delay = 0 and you’ll see that the runtime switches between goroutines. (With delay = 1ns, the switches are of course obvious.)

1 Like
12

That’s really cool! Thanks for helping me out. Much appreciated!

13

Oh and this code sample even has no blocking I/O like Print calls or similar.

Run this locally (not sure if the Playground provides more than one core tbh) and you can see a few switches between the goroutines. (On my machine, I have to run the code multiple times to see the switching. Most of the time, it really looks as if both goroutines run one after another.)

14

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.