Go compiler not using all cores

I have noticed that whenever Go needs to compile a huge project, like GTK, it takes a long time (in computer terms, it really only takes about 5 mins). The processor is usually only at 10-20% busy at the time, so I am guessing that it is only using one core. So why isn’t it using more cores? Is it because everything needs to be compiled in process, or have I not set up my environment to use more cores?

1 Like

The Go gc compiler has been modified to exploit concurrency. This permits a degree of parallelism.

GTK is wriiten in C and the Go cgo binding process runs C compilers with no parallelism.

GTK is most likely the problem.

1 Like

Thanks, so what you are saying then, is that when it compiles GoTK3, it is not just compiling GoTK3 (go), but also the entire GTK (c/c++)? And that’s the reason it takes extra time and only uses one core? Have I understood it correctly?

Rob Pike got fed up when a minor change to a large C++ program took 38 minutes to compile. He started to design Go: The Go Programming Language Promo.

Go cgo compiles elements of GTK (it’s large), and then reads the header and object files to extract type, linker, and other information. C and C++ are notoriously slow. It is also work to link GTK (it’s large) to Go.

Let us know if a large Go project, without cgo (CGO_ENABLED=0), takes a long time to compile and link.

I compile Go from source. I just watched the entire Go standard library compiling with the Go gc compiler. All 4 cores were busy, often at 100%.

Just to be clear, it is only the first time, for example in a newly created GoTK3-project (after go get github.com/GoTK3/GoTK3), or after GTK has been updated on my machine, that it takes a long time to compile. I am running Manjaro, so I get a new GTK version pretty much as soon as they are released…and then I have to compile every single GoTK3 application again (slow) once…

After that compiling is super fast.

I can live with the time loss though, it is not that important, I was just curious…

Go makes every effort to speed up compiling and linking. It caches intermediate results. If there has been no change, it uses the cached result. Otherwise, it recompiles.

For example:

  1. List the hello.go program.

  2. Run the hello.go program once to prime caches.

  3. Time and run the hello.go program: real 0m0.103s.

  4. Time and run the hello.go program using flag -a to override the cached values: real 0m2.332s.


$ cat hello.go
package main

import "fmt"

func main() {
	fmt.Println("Hello, Gophers!")
}

$ go run hello.go
Hello, Gophers!

$ time go run -v hello.go
Hello, Gophers!
real	0m0.103s
user	0m0.120s
sys	    0m0.040s

$ time go run -v -a hello.go
runtime/internal/sys
internal/unsafeheader
runtime/internal/atomic
internal/cpu
math/bits
runtime/internal/math
unicode/utf8
internal/race
sync/atomic
unicode
internal/bytealg
math
runtime
internal/reflectlite
sync
internal/testlog
errors
sort
io
internal/oserror
strconv
syscall
reflect
time
internal/syscall/unix
internal/syscall/execenv
internal/poll
os
internal/fmtsort
fmt
command-line-arguments
Hello, Gophers!
real	0m2.332s
user	0m6.517s
sys	    0m0.473s
$

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.