Hello,
I wrote a blog post about how to handle logging in the context of a reusable package: https://0value.com/about-Go-logging
Thanks,
Martin
1 Like
I wish the standard logging package was better, the fragmentation of all the improved logging packages is a nuisance.
Yeah, there’s this proposal for Go2 to make log.Logger an interface, but I wouldn’t hold my breath 
Hmm I can’t put another link in a post, but that’s issue #14205 on Go’s repo.
Very nice. I like the solution you give - just pass a function.
I think the one thing I wanted to see was that much of the time, the correct answer is “don’t log”. Succeed, or return errors. Period. Let the caller worry about logging. It should be very rare that a reusable library needs to log.
3 Likes
Heh, didn’t you just argue in another thread that Go doesn’t need a “standard” package manager and this is an area for competiton amongst many third parties.
You cannot have it both ways … 
Precisely.
1 Like
Agreed, but this is specifically for cases when you have crossed that line and decided you had to log stuff. The stdlib’s HTTP server is a good example of such a package.
A package manager can be developed without impacting how Go code needs to be written.
Whereas the different loggers have different APIs, so they cause code interoperability problems.
Basically, APIs need standardization; implementations, not so much.
I updated the post to make that clearer right from the start, thanks Nate!
Use fmt.Fprintf(os.Stderr, ...).
I’m serious. If it’s important enough to tell the user, it’s important enough to use stderr. And if your application logs so much rubbish that people complain and ask for an option to turn down the logging, that’s the problem, not a result of not choosing the perfect logging library.
2 Likes
I understand the idea of minimalism, but people will want to have the logs fit into their logging solution of choice. I think the LogFunc is minimal and simple enough while providing that added benefit and flexibility. And if you really just want to send to stderr, you still can, of course.
Note that I’m not advocating for those myriad of (sometimes overly-complex IMHO) logging solutions, I just reflected on how best to support the fact that there are that many popular options, and how as package writers we can avoid limiting the choice of package users, since they (apparently) want many of those options.
What could be simpler than logging to stderr ? Applications must not manage their own log files.
I actually disagree with this quite a bit (I did, after all, write a package specifically for managing your own log files).
Under ideal circumstances (applications run by experts, in standardized environments, with plenty of bandwidth for their devops folks), I agree… mostly.
However, many applications do not run in those ideal circumstances. Applications run on an end user’s machine, applications that run across many different platforms, applications run by non-experts, etc. In these and many other cases, sometimes making a good-enough effort to fulfill some basic needs is a good idea. That’s not to say that you shouldn’t also give the user running the application the option to write to stderr if they do fit more closely into the ideal circumstances.
“Those who do not understand Unix are condemned to reinvent it, poorly.”
– Henry Spencer
Yes, a thousand times yes.
I’ve written or helped to write two of the logging packages out there, log15, and Go kit log. This discussion has come up in those projects, and I wrote down my thoughts on this issue here: Proposal: Remove log.DefaultLogger · Issue #42 · go-kit/kit · GitHub.
I agree with some of your points, but I think there are some factual errors and perhaps a gap in the categorization of the libraries.
The article points out that logxi accepts lists of key/value pairs. The idea of logging key/value pairs is known as structured logging. Unfortunately the article does not recognize that logxi is a derivative of log15, which is a derivative of logrus and that log15 and logrus are also structured logging packages.
The logrus API is rather overloaded in the sense that it supports both Printf-style methods and structured logging. The WithFields method provides structured logging, and the Printf-style methods just format their input and use the resulting string as the value for the “msg” key.
I would be remiss if I did not point out the log package we have written as part of Go-kit. It also adopts the structured logging approach and then aims for a simple and highly composable API.
I gave a talk last year about how we arrived at the design of the Go kit log package. (slides here)
Spoiler: Go kit log defines this logging interface.
// Logger is the fundamental interface for all log operations. Log creates a
// log event from keyvals, a variadic sequence of alternating keys and values.
// Implementations must be safe for concurrent use by multiple goroutines. In
// particular, any implementation of Logger that appends to keyvals or
// modifies any of its elements must make a copy first.
type Logger interface {
Log(keyvals ...interface{}) error
}
We reworked some parts of the API since I gave the talk linked above, but the central interface is still the same.
In conclusion, I am glad there are other people who care about these issues and I hope that my thoughts have enriched the conversation.
Please no. This is a fail on Windows services, no output. I’ve also used stdin out and err to communicate between procs. I hate it when I see this.
Hi Chris, thanks for the comments! I did not expect to raise so much discussion, but that’s probably because logging is such an opinionated field! And that’s kind of the whole point of the article - the goal was not to analyze all different approaches provided by those libraries, but to find a common denominator that can be used for the (few) reusable packages that need to do logging. This is about not taking sides, or as little as possible.
As mentioned to Nate earlier (and updated in the blog post to make it stand out) I do agree that very few reusable packages should do logging.
I’m sorry if I missed that, but AFAICS the readme in both projects don’t mention this besides logrus being one of many “inspirations” for log15, and logxi comparing to them for performance. About the structured logging part, again that was not the goal of the article to cover all options of those logging libraries, but only to find how a reusable package can connect to most of them in a minimal way. I certainly did a poor job of making that clear.
That’s the better approach IMO for application-level logging - to use structured logging. The blog post doesn’t advocate against that in any way. But structured logging can (and should IMO) have a “message” field, along with other fields (process name, ID, timestamp, etc.). The Printf-style function passed to LogFunc should be to format the message to the underlying structured event. I really don’t think a reusable package should add fields to the structured logging - you want structured logging precisely because it is structured, you don’t want to allow arbitrary keys in there.
So I don’t see any problem with having a flexible LogFunc on one side (the reusable package side) and preferring structured logging on the other side (the app side).
Fair enough. The point I was trying to make about the article was that I thought it would be more accurate if it identified log15’s key/value pair style the same way that it did logxi, since they both handle their arguments the same way. Neither does any Printf-style formatting.
I guess we disagree on that point. I believe that if you are going to use structured logging then all log data should be structured, not just a few fields.
Oh gotcha, I did miss that. Will update the post, thanks.
How would you know which fields to add/set in a reusable package? If you accept arbitrary keys, you lose a lot of the benefit of structured logging - yes, it’s structured, but it’s a structured mess. And you don’t use a “message” field in your structured logging? That’s all a reusable package should write IMO.
I would say it is less of a mess than arbitrary text in a “msg” field.
No, I don’t find a “msg” field useful. This conversation came up on the Go kit slack channel just yesterday and I said this:
My experience using log15 (which has an implicit “msg” key before the first argument as you desire) is that once I log the right ​data​ in my key/value pairs, I often have little to nothing of value to put in the “msg”; but of course the library requires me to, so I end up with some extra noise.
I recommend thinking hard about what you plan to put in the omnipresent “msg” value and see if it is really universally needed.