Build with private dependencies in docker container not working reliably

I’ve had several situations in which it magically seemed to work for one single revisions. But as soon as I update revisions I repeat to get into the same troubles that always take hours to resolve.

Here’s what I have so far:

  1. Github SSH is set up and working as far as I can tell.
  2. Local build is working
  3. Go mod contains the revision I want to use

The magical couple of times it worked I had the following build process:

In a bash-script I to the following

go clean -modcache
go mod tidy
go mod download
go mod vendor

In a Dockerfile:

FROM golang:1.16-buster AS build
WORKDIR /app
COPY go.mod ./ 
COPY go.sum ./
COPY vendor ./
COPY *.go ./
ENV GO111MODULE=on
RUN go build -o /<product-name>

This continues to stop working for incomprehensible reasons with the error message:

go: github.com/.../@v0.0.3: reading github.com/.../.../go.mod at revision v0.0.3: unknown revision v0.0.3

This while local build works flawlessly and the vendor folder contains the revision, modules.txt

# github.com/.../... v0.0.3
## explicit
github.com/.../.../...

As the error keeps coming back I lost any confidence in my understanding of what I’m doing concerning the go module system.

I would be super happy if anyone could come up with a step by step guide that reliably builds go projects with private dependencies in a docker container and does that in a reproducible manner.

Thanks in advance

So, if only your problem is consume for a private dependencies, I’m thinking you need the credentials of a repository, and the GOPRIVATE configured, see this tutorial about this: How to Use a Private Go Module in Your Own Project | DigitalOcean

Thanks for the reply. I know this link and used it a couple of times. It does not yield a reliable solution to the problem. A couple of weeks later/ one release version later and I can repeat the whole process without ever knowing precisely why it stopped working in the first place, or which of the steps was the reason for it not working anymore.

So in my opinion this process (though being the official “right” way of doing it) is not solid. I suspect it’s because the error messages are missleading. In my experience this could be a pointer to something being wrong in the implementation. I mean if error messages are not being precise enough for fixing the problem effectively.

I returned doing it with a replace pointing to a local copy, thus bypassing all the unnecessary complexity with getting the revision/version to work, if this is actually the problem. I suspect it’s not.

I would prefer a module system that is as reliable and easy to use as the one of NodeJS. In my opinion the following defaults would make the module system easier to use:

  • Default for storing dependencies is the vendor folder
  • Versions are optional and the local one inside of vendor is always fallback
  • Dependencies in gopath/pkg could be an optional config

So in general the module system should be designed around a process that makes it possible to have easy to use and control (even in a container) default fallbacks in every step.