Creating a log routine for logging errors

[Glenn_Hancock]

I’m writing a REST server in go and wondering what the proper method for logging errors to a /var/log/myservice.log file might be. Specifically, with a service that can be receiving thousands of requests per minute, is there any type of buffering or anything else I need to be concerned about with go?

Can I open a single file handle and pass it around for the different threads to write to or do I need to be handling a different way?

[johandalabacka (Johan Dahl)]

The log package is safe to call from multiple go routines.

[johandalabacka (Johan Dahl)]

package main

import (
	"fmt"
	"io"
	"log"
	"net/http"
	"os"
)

func doLog(w http.ResponseWriter, r *http.Request) {
	io.WriteString(w, "Hello world")
	log.Println(r.RemoteAddr, r.RequestURI)
}

func noLog(w http.ResponseWriter, r *http.Request) {
	io.WriteString(w, "Hello world")
}

func main() {
	f, err := os.Create("test.log")
	if err != nil {
		log.Fatalln(err)
	}
	defer f.Close()
	log.SetOutput(f)

	http.HandleFunc("/", noLog)
	http.HandleFunc("/log", doLog)
	fmt.Println(http.ListenAndServe(":8080", nil))
}

1000 calls 100 concurrent

ab -n 1000 -c 100 http://localhost:8080/
Requests per second: 5188.09 [#/sec] (mean)

ab -n 1000 -c 100 http://localhost:8080/log
Requests per second: 4097.35 [#/sec] (mean)

Logging takes a bit of time but not much relatively other operations like sql access etc.

package main

import (
	"bufio"
	"fmt"
	"io"
	"log"
	"net/http"
	"os"
)

func doLog(w http.ResponseWriter, r *http.Request) {
	io.WriteString(w, "Hello world")
	log.Println(r.RemoteAddr, r.RequestURI)
}

func noLog(w http.ResponseWriter, r *http.Request) {
	io.WriteString(w, "Hello world")
}

func main() {
	f, err := os.Create("test.log")
	if err != nil {
		log.Fatalln(err)
	}

	w := bufio.NewWriterSize(f, 1024*65)

	defer f.Close()
	log.SetOutput(w)

	http.HandleFunc("/", noLog)
	http.HandleFunc("/log", doLog)
	fmt.Println(http.ListenAndServe(":8080", nil))
}

By adding a buffer to the writer is the difference almost zero. However then you will get your log in larger chunks which can be a problem if you want to look at it in realtime.

[Glenn_Hancock]

Thanks for the explanation. I’ve gotten the logger running now and seems to be working well. I obviously don’t write logs out for every transaction, only when there are problems so it shouldn’t really be much of a concern.

thanks again,

Glenn