Log - simple, configurable and scalable Structured Logging for Go

Releases
1

Releasing log, been using it in production for a while and is ready for broader use.

Some Highlights:

  • Logger is simple, only logic to create the log entry and send it off to the handlers and they take it from there.
  • Sends the log entry to the handlers asynchronously, but waits for all to complete; meaning all your handlers can be dealing with the log entry at the same time, but log will wait until all have completed before moving on.
  • Ability to specify which log levels get sent to each handler
  • Built-in console + syslog
  • Handlers are simple to write + easy to register
  • Logger is a singleton ( one of the few instances a singleton is desired ) so the root package registers which handlers are used and any libraries just follow suit
1 Like
2

Hi,
Great work. I am looking at log libraries at the moment - in what way do this one differ from Logrus?

For now I only need to log to a local file, but will be looking at sending logs to something like Logstash, I guess this will be possible with using handlers?

Also, by sending these to the handlers won’t slow down the normal flow of the application ?

3

Hello @pieterlouw

Thanks for the question, I highly recommend reading the README as it outlines it’s differences and unique features; anyways here is a quick summary.

Basically the biggest differences are:

  • that this library logs to multiple handlers asynchronously, yet waits for all handlers to have handled the log to guarantee logs complete before proceeding; currently most loggers log to multiple handlers synchronously much like the std library does with it’s multiwriter implimentation see here line 42 and so this library should scale better as more handlers are added.

  • you can easily specify which logs levels each handler should handle while adding the handler to the log package.

  • library is designed with centralized logging in mind, also see README

basically wanted to make the log library as easy as possible to use.

Yes you should be able to log to logstash using the builtin http handler just set the encoding to "application/json" and then set your own FormatFunc to just marshal the JSON. ( if you do end up choosing this library just let me know and I can help )

As for slowdowns, it should not slow you application down; although it is possible for the handlers buffer to block, that is why I allow for the size of the buffer and number of workers listening on the buffer to be adjusted so that you can adjust if that were ever to happen; I run this in a production webapp and have never seen any sort of slowdown; just see the benchmarks in the README compared to logrus

NOTE: in no way am I saying logrus is bad in any way, it is a great library, I merely created this library for better ease of use.

1 Like
4

Hi @joeybloggs

Thanks alot for your in-depth answer.

I’m not sure if you are familiar with go-kit, if you are, do you see your library being able to be used as the logging middleware in go-kit applications?

Also, would you say that the library follow the 12Factor “rule” of Treat logs as event streams ?

5

hey @pieterlouw

yes I am familiar with go-kit, actually been investigating microservice frameworks recently, not too in depth but enough to know that yes my library could be used in the logging middleware of go-kit applications. The only difference with is that my log library registers itself as a singleton (one of the handful of time that a singleton is desired) and so would not have to pass it into/made accessible by using a decorator.

i.e. wouldn’t need the outer function loggingMiddleware to pass in the log

func loggingMiddleware(logger log.Logger) Middleware {
    return func(next endpoint.Endpoint) endpoint.Endpoint {
        return func(ctx context.Context, request interface{}) (interface{}, error) {
            logger.Log("msg", "calling endpoint")
            defer logger.Log("msg", "called endpoint")
            return next(ctx, request)
        }
    }
}

As for the 12 Factor rule, I would say yes except for the following line "Instead, each running process writes its event stream, unbuffered, to stdout" in my library you can setup the logger, using the built-in console logger, to output to stdout but that is up to the developer and isn’t guaranteed by the library.

other than that yes I would say that follows the 12 factor “rule”

P.S. the console logger also has a method RedirectSTDLogOutput(b bool) that you can set to true to redirect the std library logs to the log library itself. It allows for any errors/warning produced by code out of your control to still be logged.

1 Like
6

Thanks for your answers, it helps to ask the questions directly as it’s difficult to tinker with the library because of time constraints.

7

no problem, if you think of any other question just let me know.

8

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.