Go compiler not using all cores

Getting Help
1

I have noticed that whenever Go needs to compile a huge project, like GTK, it takes a long time (in computer terms, it really only takes about 5 mins). The processor is usually only at 10-20% busy at the time, so I am guessing that it is only using one core. So why isn’t it using more cores? Is it because everything needs to be compiled in process, or have I not set up my environment to use more cores?

1 Like
2

The Go gc compiler has been modified to exploit concurrency. This permits a degree of parallelism.

GTK is wriiten in C and the Go cgo binding process runs C compilers with no parallelism.

GTK is most likely the problem.

1 Like
3

Thanks, so what you are saying then, is that when it compiles GoTK3, it is not just compiling GoTK3 (go), but also the entire GTK (c/c++)? And that’s the reason it takes extra time and only uses one core? Have I understood it correctly?

4

Rob Pike got fed up when a minor change to a large C++ program took 38 minutes to compile. He started to design Go: The Go Programming Language Promo.

Go cgo compiles elements of GTK (it’s large), and then reads the header and object files to extract type, linker, and other information. C and C++ are notoriously slow. It is also work to link GTK (it’s large) to Go.

Let us know if a large Go project, without cgo (CGO_ENABLED=0), takes a long time to compile and link.

I compile Go from source. I just watched the entire Go standard library compiling with the Go gc compiler. All 4 cores were busy, often at 100%.

5

Just to be clear, it is only the first time, for example in a newly created GoTK3-project (after go get github.com/GoTK3/GoTK3), or after GTK has been updated on my machine, that it takes a long time to compile. I am running Manjaro, so I get a new GTK version pretty much as soon as they are released…and then I have to compile every single GoTK3 application again (slow) once…

After that compiling is super fast.

I can live with the time loss though, it is not that important, I was just curious…

6

Go makes every effort to speed up compiling and linking. It caches intermediate results. If there has been no change, it uses the cached result. Otherwise, it recompiles.

For example:

  1. List the hello.go program.

  2. Run the hello.go program once to prime caches.

  3. Time and run the hello.go program: real 0m0.103s.

  4. Time and run the hello.go program using flag -a to override the cached values: real 0m2.332s.

$ cat hello.go
package main

import "fmt"

func main() {
	fmt.Println("Hello, Gophers!")
}

$ go run hello.go
Hello, Gophers!

$ time go run -v hello.go
Hello, Gophers!
real	0m0.103s
user	0m0.120s
sys	    0m0.040s

$ time go run -v -a hello.go
runtime/internal/sys
internal/unsafeheader
runtime/internal/atomic
internal/cpu
math/bits
runtime/internal/math
unicode/utf8
internal/race
sync/atomic
unicode
internal/bytealg
math
runtime
internal/reflectlite
sync
internal/testlog
errors
sort
io
internal/oserror
strconv
syscall
reflect
time
internal/syscall/unix
internal/syscall/execenv
internal/poll
os
internal/fmtsort
fmt
command-line-arguments
Hello, Gophers!
real	0m2.332s
user	0m6.517s
sys	    0m0.473s
$
7

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.