Problem with remote imports from a module package

[mustikkakeitto (Alexander)]

Hi, here is the problem:
I did refactor the code of my project and moved a specific functionality to a local package.
But now I’m encountering problems importing remote GitHub packages from that package.

Here is the structure om my project (it placed just “somewhere”):

login_watch
│   build4linux.cmd
│   Dockerfile
│   go.mod
│   go.sum
│   main.go
│
├───bin
├───log
└───pkg
    └───analyzer
            series.go
            series_test.go

That is, my app is a module named “github.com/mustikkakeitto/login_watch” in the login_watch folder somewhere on the disk, and it has main.go file:

package main

import (
  "container/list"
  "fmt"
  "github.com/mustikkakeitto/login_watch/pkg/analyzer"
  "time"

  "encoding/json"
  "log"
  "net/http"

  "sync"

  "github.com/gorilla/mux"
  rotatelogs "github.com/lestrrat-go/file-rotatelogs"
) 
// etc

main.go imports the local package as “github.com/mustikkakeitto/login_watch/pkg/analyzer”.
That package, in its turn, imports 3rd-party remote package golang-levenshtein/levenshtein" from the GitHub,
here is the contents of login_watch/pkg/analyzer/series.go:

// Package analyzer implements functions to detect series
// built from the particular phone numbers
package analyzer

import (
  "container/list"
  "fmt"
  "time"

  "github.com/texttheater/golang-levenshtein/levenshtein"

  "log"
)
// etc

So now, when I start the build / run command, I see this:
go run .
pkg\analyzer\series.go:10:2: cannot find package

I just don’t understand, why the imported library package could not be found?
It worked pretty well, when all the functionality from pkg\analyzer\series.go resided in the \main.go …

Here is my go.mod file, in the root of my project:

module github.com/mustikkakeitto/login_watch

go 1.16

require (
  github.com/gorilla/mux v1.8.0
  github.com/jonboulle/clockwork v0.2.2 // indirect
  github.com/lestrrat-go/file-rotatelogs v2.4.0+incompatible
  github.com/lestrrat-go/strftime v1.0.4 // indirect
  github.com/pkg/errors v0.9.1 // indirect
  github.com/texttheater/golang-levenshtein/levenshtein v0.0.0-20200805054039-cae8b0eaed6c // indirect
)

And some other info, for the record:
go version go1.16.4 windows/amd64
GOPATH=C:\Users\Alex\go
Environment variable GOROOT not defined
Environment variable GO111MODULE not defined

Any help would be appreciated!

[christophberger]

Hi @mustikkakeitto, welcome here.

The error you get is indeed strange.
I did a quick test by creating a main and an analyzer package:

main.go:

package main

import (
	"fmt"

	"github.com/mustikkakeitto/login_watch/pkg/analyzer"
)

func main() {
	fmt.Println(analyzer.Dist())
}

and pkg/analyzer/series.go:

package analyzer

import "github.com/texttheater/golang-levenshtein/levenshtein"

func Dist() int {
	return levenshtein.DistanceForStrings([]rune("a"), []rune("aa"), levenshtein.DefaultOptions)
}

then I ran

go mod init github.com/mustikkakeitto/login_watch
go mod tidy

in the root dir to generate go.mod as:

module 

go 1.16

require github.com/texttheater/golang-levenshtein/levenshtein v0.0.0-20200805054039-cae8b0eaed6c

and go run main.go returns 1 without any errors.

What you could try:

• run go mod tidy
• run go mod download github.com/texttheater/golang-levenshtein/levenshtein
• run go mod graph and verify if the output looks expected
• if neither of the two helps, run go clean -modcache (but be aware that this clears the whole module cache, and future compiles might take longer until the cache is filled again)

[mustikkakeitto (Alexander)]

Hi @christophberger, thank you for the reply!

To cut a long story short, the problem was caused by the UTF-BOM header ( EF BB BF ) of my version pkg/analyzer/series.go file - seems it somehow prevents the compiler to parse the source file.

I can’t recall how exactly I created this particular file, but unlike the all others it got the header.

I’ve just reimplemented your quick test in a separate folder and ensured it works.
Then I’ve incrementally added chunks of the original code to the series.go and tested it every single time, and finally did the byte-to-byte comparison with the original version.

God knows, how long would it take for me to figure the damn thing out without your help!

Thanks a lot!

[christophberger]

Oh wow, that’s a really tricky bug indeed! Glad to hear that you found & fixed it. I would never have imagined some nasty BOM as the source of this problem.

A BOM is quite useless for UTF-8, yet it is permitted in general, and since Go 1.1 (no typo) it is also permitted for Go source files. So nothing wrong from your side.

I found a similar issue in the GitHub mirror of the Go repo. (Same cause, different error message.) According to a comment there, running go fmt removes the BOM automatically.

And the problem seems to be very close to getting resolved.