I created a multi-threaded twinprimes sieve
in Go, to compare with other implementations.
I realize Go doesn’t have true parallelism, but concurrency using Go routines.
The problem I’ve encountered is that it appears each Go routine retains use of the memory it uses until all the threads/processes end, instead of releasing memory when its particular process ends.
This causes the program to eat up more and more memory as the number of Go routines increase.
As the inputs get larger more memory is used until the program bombs from exceeding available memory.
Below is the code section at its end for performing the Go routines.
sums := make([]uint, pairscnt)
lastwins := make([]uint, pairscnt)
var wg sync.WaitGroup
for i, r_hi := range restwins {
wg.Add(1)
go func(i, r_hi int) {
defer wg.Done()
l, c := twins_sieve(r_hi, kmin, kmax, kb, start_num, end_num, modpg, primes, resinvrs)
lastwins[i] = l; sumss[i] = c
fmt.Printf("\r%d of %d twinpairs done", (i + 1), pairscnt)
}(i, r_hi)
}
wg.Wait()
Below is a gist of the total code.
You can compile and run it and see its memory use, using e.g. htop
, as inputs increase.
Even though I have 16 GB of memory, it gets eaten up fast as inputs values increase.
Is there a way for the Go routines to immediately release memory when they cease processing?