[gophersource] Building Go from Source

Technical Discussion
1

This is part of a larger group activity, [gophersource], where you can join in and learn how to contribute to upstream Go and other OSS projects.

  1. Find the docs to build go from source :white_check_mark: https://golang.org/doc/install/source
  2. Go is compiled with … Go! :exploding_head: I need to decide if I want to bootstrap a go compiler, or use the Go that’s already on my computer. I’m lazy so I’m going with the latter. :white_check_mark:
  3. $GOROOT_BOOTSTRAP is an environment variable that must be set to the location of the Go compiler that I want to use. Since I’m using the regular Go installation that I already had on my mac, I set it using export GOROOT_BOOTSTRAP=$(go env GOROOT). I tested that out by running $GOROOT_BOOTSTRAP/bin/go and making sure I saw the Go help text.
  4. Clone the Go source code git clone https://go.googlesource.com/go. I am not sure if it needs to be in the GOPATH or not (I’m assuming no?).
  5. Change to the go source and run git tag to admire how many tags there are and see what version I can start building off of. I picked the latest git checkout go1.11beta1.
  6. Change to the src/ dir in the repo and then run ./all.bash. Immediately regret deciding to do this on my mac book air.
  7. One of the tests failed, for os/signal with some odd timeouts. I’m going to blame that on my old laptop and hope for the best!
  8. Since the test failed, I am bypassing them and just doing a build+install with ./make.bash.
  9. Switch my current shell session from using a stable go to the one I just built
    $ go version
go version go1.10 darwin/amd64
$ which go
/usr/local/go/bin/go

$ export PATH=/Users/carolynvs/src/go/bin:$PATH
$ which go
/Users/carolynvs/src/go/bin/go
$ go version
go version devel +a12c1f26e4 Tue Jun 26 20:00:51 2018 +0000 darwin/amd64
  10. OK, now how do I tweak it so that the new vgo prototype capabilities are enabled? Found the Pull Request (Google likes to call them CL’s) here: https://go-review.googlesource.com/c/go/+/118095

… a brief interlude while I enjoy my weekend …

Weather update: it’s still way too hot in Chicago, so the hacking continues!

So I’ve learned that the vgo prototype was merged into upstream Go, but it’s turned off. I’ve made a branch with it turned back on. The prototype’s code lives in src/cmd/go/internal/vgo and hasn’t yet been fully hooked up to all the commands. Seems like go build works, and uses the prototype, but stuff like go list -m isn’t wired up yet.

I made a sample repo to try out my spiffy local build of Go. The go.mod file exists to indicate to go that we want to use all the cool new stuff. I left it empty so that when I run go build that it will detect my dependencies and populate the file for me.

$ go run main.go
vgo: resolving import "github.com/pkg/errors"
vgo: finding github.com/pkg/errors (latest)
vgo: adding github.com/pkg/errors v0.8.0
hello world!

HOT DAMN! :dancer:

We’ve built Go from source, edited the source code to enable the vgo prototype, tweaked our PATH so that we can try out our custom build, and seen some output that makes me think I did that right. :tada:

2 Likes
2

If you are following along (I hope you are!), we’ve figured out how to build from source but haven’t broken down what’s going on under the hood. I’d like to know a bit more about:

  1. What’s going on when I run ./src/make.bash? It seems like it builds and then rebuilds the code?
  2. Why can’t I run all the tests successfully?
  3. It seems that I don’t need to clone the Go source into a GOPATH. Is that right or will I have regrets later? :thinking:

If you have more questions, or even better have some answers for me, please share!

5

The purpose of this thread, and all of the ones marked [gophersource] is to learn how to become Go contributors. That is why I followed the instructions from the official Go documentation, and used git: so that as we make changes, we can submit pull requests upstream.

6

TIL that the vgo prototype isn’t completely wired up into the main go command upstream yet. So if you want to try out go get with the prototype, you will still have to use the vgo prototype directly for now. Seems like only go build is wired up at the moment.

I’ll report back when that changes!

1 Like
7

Hi,

Just to let you know I managed to build and pass tests with go1.11beta1. Yet, I wasn’t able to activate vgo in the build. How did you manage that ?
My project was installed anywhere in file system and it seems to cause no consequences.
Did you know about that repo ? http://vgo.readthedocs.io/en/latest/installation.html
I’ve just installed it right now, I’ll try it further later. Have a nice day.

8

That’s a different vgo. See https://github.com/golang/go/wiki/vgo for more information on the intended one including articles and videos. https://www.youtube.com/watch?v=F8nrpe0XWRg is a good introduction.

9

@imatmati I removed the hard-coded feature flag disabling the vgo prototype. The change is very small

Here’s my branch GitHub - carolynvs/go at go1.11beta1-vgo-enabled

10

Looks like Russ is done wiring up the vgo prototype into upstream Go, now to be called “go module” support. :tada:

https://groups.google.com/forum/#!msg/golang-dev/a5PqQuBljF4/61QK4JdtBgAJ

So I am going to play with it this weekend and see if that can completely replace my patch, and let us keep chugging along with trying it out.

11

After checking out the master branch and pulling latest, I can now use “Go modules” (which is the long term name for the features added by the vgo prototype)! :nerd_face:

A lot of the interesting code is in src/cmd/go/internal/modload/init.go. I recommend starting there to figure out when Go modules are enabled and as a way to track when they are being used by the other code in the go toolchain (such as get or build). Just looking for who is calling modload.Enabled or just searching for the term modload. seems to be a good way so far to see where special “go module” features have been added throughout the codebase.

I’m going to write up a blog post today that takes my notes and gives people the “shortest path to success” and link back to it here.

3 Likes
12

Building Go from source on Linux (Ubuntu 18.04)

This is the way I’ve built/set up Go from source on a brand new system (that didn’t have Go installed previously) following the official docs.

  1. Create an empty $HOME/go directory

  2. Download and extract the go1.4 source distribution to $HOME, and rename the directory you just unpacked to go1.4.

  3. To build the bootstrap toolchain run the following:

$ cd $HOME/go1.4
$ ./make.bash

(setting CGO_ENABLED=0 was not necessary for me)

  1. Fetch the Go repository and install the desired version:
$ cd /usr/local/
$ git clone https://go.googlesource.com/go

To install and change between versions, checkout the stable version’s branch e.g git checkout go1.10.3 or stay in master for the development branch, and run:

$ cd /usr/local/go/src/
$ ./all.bash

It will take some time to build and run all the tests, once this is done add the following to your ~/.profile, ~/.bash_profile or whichever file you use to be executed when you start a shell.

export GOROOT=/usr/local/go
export PATH=$PATH:/usr/local/go/bin
export PATH=$PATH:/home/[YOUR USER]/go/bin

apply the changes:

$ source ~/.profile
  1. Verify the version installed
$ go version
go version devel +f17220c208 Tue Jul 17 05:41:06 2018 +0000 linux/amd64

VGO / Go Modules

If you have the master branch installed these will be available by default since CL 123580. For more information on usage run go help modules on your CLI :grin:

13

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.