Deadlock in text processing (beginner)

Hello,
This is really a beginner question, but I cannot understand why I get a deadlock. This code is supposed to cut a string array in four pieces, send them through a goroutine to a frequency counter and then print the result.

// Split load optimally across processor cores.
func WordCount(text string) {

	//doing text processing here (omitted)

	numberOfGroups := 4
	piece := len(sentence) / numberOfGroups
        //buffered channel, but it does not work better with an unbuffered one
	wordChannel := make(chan map[string]int, numberOfGroups)

	wg := new(sync.WaitGroup)
	wg.Add(numberOfGroups)

	for i := 0; i+piece < len(sentence); i += piece {
		go processToCounting(sentence[i:i+piece], wordChannel, wg)
	}

	wg.Wait()
	close(wordChannel)
	fmt.Print(<-wordChannel)

}

func processToCounting(textSlice []string, wordChannel chan map[string]int, wg *sync.WaitGroup) {
	freq := make(map[string]int)
	for _, v := range textSlice {
		freq[v]++
	}

        //does not work with defer either
        wg.Done()
	wordChannel <- freq
}

Hi @Dj_Sparks,

You need to put wg.Wait() inside of a goroutine so it doesn’t block. For example, this is how you would commonly see it being done:

go func() {
	wg.Wait()
	close(wordChannel)
}()

fmt.Print(<-wordChannel)

Thank you. Do you mind to explain why?

Sure. In your example you are adding numberOfGroups to wg. However it’s possible that this line for i := 0; i+piece < len(sentence); i += piece is exectued less times then numberOfGroups times (4), you are not ever returning a 4th wg.Done().

For example, you could instead do something like this and only add to the wg counter every loop:

for i := 0; i+piece < len(sentence); i += piece {
	wg.Add(1)
	go processToCounting(sentence[i:i+piece], wordChannel, wg)
}

But you would still probably finish with something like this:

go func() {
	wg.Wait()
	close(wordChannel)
}()

for val := range wordChannel {
	fmt.Println(val)
}

Edit: Or in your case something like this for the final counts:

go func() {
	wg.Wait()
	close(wordChannel)
}()

frequencyMap := map[string]int{}
for val := range wordChannel {
	for k, v := range val {
		frequencyMap[k] += v
	}
}

fmt.Println(frequencyMap)

I hope that made sense :stuck_out_tongue:

1 Like

Thank you, I am vers grateful. That’s true that go is vers confusing when one start. And in other exercices I have the wg.Add(1) inside of the loop, so I understand what you are saying.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.