The Go community is the one popular language community that doesn’t have packages/libraries released with versions. The most common case is no version followed by versions that are out of date (not useful). After that we get into the case of SemVer in gopkg.in.
Can we please start doing real releases via SemVer. The way we don’t do it is hurting the community and a point of frustration for many devs. I (and many others) believe it’s hurting the adoption of Go.
this is a good point, when i argue the case for using Go, this is the first thing people point out, distinct lack of versioning of packages, i know this is a can of worms, just reporting the feedback i get when trying to promote Go. Vendor is just the beginning, i think its a good step forward, you still need tools like glide to lock versions…
for me right now, glide works ok, can target specific versions of external packages etc…
I think one reason this doesn’t have much traction is that it isn’t supported by the tools most people use. I can version all I like if noone actually looks at or uses the version number…
I don’t really understand your “state of go versioning” section:
9 of the top 20 most imported according to Go Search had no release versions at all. When I look at other lists on Godoc or Go Search these ratio of only about half having versions holds up. To go further I looked at GitHub Trends for Go packages (excluding applications written in Go) and found 75% for the past month didn’t have release versions.
Of those that have release versions many are out of date. It’s not unusual to have the latest release be over a year old. Sometimes closer to two years. There can be many, sometimes hundreds of, commits since then. While these versioned releases exist they are essentially useless.
Those who are using gopkg.in. These really fall into two categories. Those who are doing incremental releases and those who just have a major branch you need to follow. The latter is similar to not versioning except you have the major version API break handled. The former is great and I like to see it.
Some have tags that have meaning only to that project. Are these releases? What do they mean? I don’t know because it’s not obvious. They appear to be one offs.
Some use release versions, keep making releases, and I’m overjoyed to see them. It makes me smile when I see packages like beego and logrus releasing versions.
2.) Are you sure that HEAD is not a separate logical “version”? Are you sure that they didn’t just stop using github’s “release” feature?
3.) What’s wrong with just having v1, v2, etc? Does it matter? It keeps you from breaking people, it keeps your code in the same place so it’s easy to find… I do this with, for example, lumberjack.v2 - gopkg.in/natefinch/lumberjack.v2 … there’s a v1. no, there’s no v1.1 or 2.1. Why does that matter?
4.) So, ask those projects to update their wiki? What does that have to do with Go at all?
5.) You really need to define what you mean by “releasing versions”.
Oh, I know about the tools, I’ve tried all of them. I’m just saying it’s not the norm at the moment. But I mostly agree with you on the general sentiment. The whole “we don’t need versions” or, worse, “versioning can never be solved 100% in the general case, so we shouldn’t try” never sat well with me.
HEAD isn’t a version as such as it moves with every commit. If I ask you what version you saw the bug on, “HEAD, a couple of months ago” doesn’t tell me much. I understand that you mean that it doesn’t matter as the API should stay compatible for ever and ever for a given import path, but that doesn’t really make it a version.
Go libraries and programs do not have version numbers in the way it is commonly understood by our counterparts in other languages communities. This is because there is no formalised notion of releasing Go source code. There is no recognised process of taking an arbitrary VCS commit hash and assigning it a version number that is meaningful for both humans and machines.
I think this is what Matt means by “release versions”
I certainly have. I’ve even discussed aspects of it with you on the mailing list and in the bug tracker at times. I like it. I don’t use it yet on any larger project, mostly because they predate gb and already have a complex enough build process that adding gb at this stage doesn’t simplify it. I think it’s a step forward from the default Go build process though.
But doesn’t this just describe the current state of the ecosystem, that @mattfarina suggests we change? There isn’t anything essentially different about the Go language that makes versioning of code impossible while it’s possible in Rust, Python, Javascript and so on, is there?
Of course, making versioning not suck is a huge problem, but I think it’s weird to approach things from the angle that “Go (the language) does not have versioning” when it’s just that “Go (the current tooling) doesn’t understand versioning”.
But this is exactly my point – Go package do not have versions.
You cannot point a tool, or a human for that matter, to the repo of a go package and ask questions like:
what versions of this package are available ?
we depend on this package, are we using the latest stable release ?
I’ll agree that for a few packages, a human can look at the list of tags, or possibly some text release notes, or a changelog.txt and have a pretty good idea at some of those questions. But for a computer; no chance.
This is why I said that I do not believe that Go packages have versions today, because none of the properties required to answer these questions exist in a standardised way that others can build tools on top of.
@NateFinch I understand that I’m proposing something different from what many Go developers have been doing. Let me try to explain a little more.
First, I’m not asking the Go team to make a change. Go itself is already versioned and it’s ~ SemVer (even if they don’t state it). It’s not like the long tail of developers is building Go from the latest commit and building applications with that.
Software releases are something I shouldn’t define. This is a fairly well defined area. For example, see the article on wikipedia for software versioning. A commit id may be a revision but that’s different from a release version.
By using HEAD we’re communicating to package users a few things:
We’re not going to make API breaking changes or we’re going to disregard communicating them.
Commit ids in VCS like Git are hashes. They aren’t incremental and can’t convey meaning. That means if you have two packages that depend on the same 3rd one and all you have is a commit id you can’t tell which which version you should be using. Were there additions to an interface or type that matter? Was there an API breaking change? No meaning is conveyed with commit ids.
Developer experience matters. It’s the difference between happy and enabled developers and those looking for what’s next. From a developer experience standpoint there a bunch of common practices that are used by all the modern languages. By not following them and not having a well communicated alternative solving their use cases we’re saying we aren’t trying to meet their needs. Developers have openly said this is causing them to avoid Go. Other developers who use Go are getting increasingly frustrated.
I’ve used NuGet for C#, Ruby gems, Maven for Java, and Leiningen for Clojure’s packages. They’re all a pleasure to use, especially Leiningen, and, IMO, better than Go’s current state. However, those languages have something in common that is radically different from Go’s model:
They’re all either pre-compiled to an intermediary binary format that is already cross-platform, or
They’re interpreted
So, yes, I can upload foobar.gem or whatever and someone can pull it. But how would that work for Go? Do I have to upload a binary package for all possible platforms and architectures? Compiled under all possible Go compilers? Do I just zip a snapshot of the source and upload that and let you compile it - how’s that any different than pulling a tagged Git commit that you can do already?
Go is closer to C/C++, and those don’t really have “package servers” either, because, well, they can’t. The compilation/linking/execution model doesn’t allow you to create a binary blob that can be linked to by all other arbitrary binary blobs unless you lock down various aspects, ala distro RPM/DEB servers that are locked down to a certain architecture and specific X/Y/Z system libraries.
@BeerIsGood You bring up an interesting point. I’ve been studying how those other systems work in practice. How they do caching, how they build things, where they store them, and so forth.
For example, take something like Cargo (used with Rust) or Composer (PHP) and you’ll see that the central server is just a metadata store. Source is downloaded as needed from the root. This works for compiled or interpreted just fine. Compiling or converting to an intermediate happens locally.
The what is more important than the how. They are trying to do reproducible builds and handle nested dependency trees. Those are in support of using release versions for security, clearly communicating changes to users, and making it easy to work with them.
C/C++ are older languages predating the modern trends in package handling. That’s why I compare Go with other newer languages. This isn’t a matter of interpreted v compiled or static v dynamic. Modern languages have developed toolchains to easy common actions.
People are trying to bring these ideas to languages like C. For example, see clibs. This appears to be more cultural and less technical.
For all the other reasons you’ve already stated, doing releases and potentially using semver can help developers in many aspects.
However reproducible builds require either a non-changing tag, hash, or copied source code. This can be recorded in addition to release tags. If a build system uses semver ranges 3.2.x it gets in the way of reproducible builds potentially. Of course, there are ways to get both. However doing releases (possibly with semver) and reproducible builds don’t appear to be related.
I agree that releases and something like semver can help developers manage conflicts that may arise when dealing with conflicting dependencies.
First, there are release versions. Release versions in the form 1.2.3. Essentially, Semantic Versioning.
Second, build tools can leverage semantic versions and semantic version constraints or ranges (e.g., 3.2.x). Using 3.2.x is really short hand for >= 3.2.0, < 3.3.0. Package managers can use ranges to try and fetch the best possible version.
@kardianos you are right to say that using version constraints or ranges doesn’t lead to non-reproducible builds. In fact, they can lead to non-reproducable builds when new versions come out.
The way a majority of these package managers work is they allow you to lock to a version. For example, Cargo and Composer generate lock files pinning the codebase to a specific version. This is the part of the package manager tooling that sets up reproducible builds.
This topic is really tangentially related to release versions using semantic versioning. They are separate but package managers tend to work with both.
Right now I’m simply asking that those who write packages do release versions as semantic versions.
The only thing that is important is a reproducible build. Structuring projects to support this must be reasonable, teachable and simple. I think between the new vendor flag and govendor this is achievable today. I think gb makes this achievable today as well.
But there are guidelines you need to follow.
An import path must not be required to change when changing versions.
All the code for a project must belong in the project source tree.
Use a vendor folder for 3rd party packages.
Use the vendor flag/gb so imports don’t require rewrites.
Packages must avoid adding versioning in their repo name.
Dependencies must be flattened out into a single vendor folder per project/repo.
Packages can have their own vendor folder but they must be merged into the project/repo folder when vendored.
There can only be a single version of any given package per project/repo.
I think versioning can help in two areas.
I need to know when merging vendor folders together if I am dealing with different versions of the same dependency or not. You could use commit ids but they are too granular.
When loading code into a vendor folder it would be nice to specify a specific version. Again, I feel commit ids are too granular.
I would love to see the govendor and gb tools implement the same standard for tagging a repo with a version and leveraging that in their respective tooling. Once something is in place, it has an opportunity to set standards.
Is it possible to do SemVer releases and support users of go get by publishing incompatible changes to a new path? My git-fu is not strong enough to answer this question on my own.
If it is not possible to support both, then it should be clear that this is a request to abandon go get users.
Are owners of the golang.org/x/ packages going to do SemVer releases?
I’m just saying it’s not the norm at the moment. But I mostly agree with you on the general sentiment. The whole “we don’t need versions” or, worse, “versioning can never be solved 100% in the general case, so we shouldn’t try” never sat well with me.