Vendoring and multiple versions of the same library

Technical Discussion
1

I have been using godep for managing project dependencies. Before 1.6 or 1.5 (without GO15VENDOREXPERIMENT set) godep created a Godeps folder that included all the vendored packages. Additionally it made sure that if the main application and its vendored packages use a library they both use the same version (paths rewritten to be the same) of that library.

With the new vendor feature the rewriting of the paths is not necessary. However the issue I’m running into is that multiple versions of the same library may now cause problems.

For example I have a library that uses glog for logging. If I import that library and also use glog in the main application I see a panic because a flag is redefined. There are other examples of this. For example only a single version of database/sql library can be imported.

What is the best way to handle these issues? One obvious way might be not to vendor packages for libraries but I’m not sure if that is reasonable.

2

When writing an application (a package main), all transitive vendored dependencies should be flattened into a single vendor folder so that only one copy of each may be built into the application.

1 Like
3

Thanks for the reply. However can you please explain how this can be achieved?

If I have a library called mylib and both mylib and main application (myapp) import the glog package then I’m not sure how this would work.

myapp has a vendor directory which includes mylib and glog. Therefore myapp would use the glog package in its vendor directory but mylib would either use glog in it’s own vendor directory (if it exists) or it would use the package in GOPATH (if it doesn’t exist). Either way the glog that myapp uses would be different than mylib.

4

Assuming the following directory structure under GOPATH:

src/
    github.com/golang/glog/
    myapp/
        myapp.go  // imports glog and mylib
    vendor/
        github.com/golang/glog/
        mylib/
            mylib.go  // imports glog

Then my reading of https://golang.org/s/go15vendor indicates that both myapp and mylib will get built with the version of glog inside the vendor folder and neither will use the other copy of glog outside the myapp tree.

If there was another vendor folder inside the vendored copy of mylib, that will cause problems like you are worried about.

1 Like
5

To clarify further: libraries should not vendor dependencies. Projects (“package main”) should.

1 Like
6

Thanks. There was another conflict with another imported library importing another version of glog that was causing a panic but was not showing up in stack trace leading me to believe the libraries would not automatically use the vendor directory.

I can confirm that it works as you indicated above.

7

I would argue that libraries should also be able to vendor packages. However it would be the job of the package manager to make sure that both libraries and the main application are using the same version of a library.

As it stands right now however it seems that we have to remove vendored packages from a library before importing it to the vendor directory of the main application.

8

When the same package is in multiple locations you can end up with 2 problems:

  1. Each location will end up in the resulting binary.
  2. Instances created from the package in one location are not compatible with the same package in another location.

Unless there is a special reason, and you know what you’re doing, libraries should not store anything in the vendor/ folder.

Tooling needs to be in place that can find the packages and put the right ones in the top level vendor/ folder. If you have to deal with different versions of a package that usually handled, as in all the other modern languages, with a manifest file and lock file at the top level. That’s what we’ve chosen to adopt with Glide.

1 Like
9

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.