Database entry of Logrus logs

[Nirmal_Singh]

Hi currently I am writing logs to a text file but I am facing problem to save error logs to MySql Db.

I have used the following code to write logs into file

func (hook ContextHook) Fire(entry *log.Entry) error {
pc := make([]uintptr, 3, 3)
cnt := runtime.Callers(6, pc)

for i := 0; i < cnt; i++ {
	fu := runtime.FuncForPC(pc[i] - 1)
	name := fu.Name()
	if !strings.Contains(name, "github.com/Sirupsen/logrus") {
		file, line := fu.FileLine(pc[i] - 1)
		entry.Data["file"] = path.Base(file)
		entry.Data["func"] = path.Base(name)
		entry.Data["line"] = line
		break
	}
}
return nil

}

Please help me to write logs into the Mysql Db.Thanks in advance.

[nathankerr]

A suitable hook using the database/sql package would look something like:

type dbHook struct {
	*sql.DB
}

func (db dbHook) Fire(e *logrus.Entry) error {
	_, err := db.Exec("insert into log (time, level, message) values (?, ?, ?)",
		e.Time,
		e.Level,
		e.Message,
	)

	return err
}

Hopefully this is enough to get you started.

[Nirmal_Singh]

Thanks,

Can you please tell me how to call this hook?

[Nirmal_Singh]

Also can you please tell me how to save these logs to redis also using logrus time

[nathankerr]

I find writing little programs helpful in figuring out how to do something. My answer to your original question was discovered by writing this little program:

package main

import (
	"database/sql"
	"log"
	"time"

	_ "github.com/mattn/go-sqlite3"
		"github.com/sirupsen/logrus"
)

type dbHook struct {
	*sql.DB
}

func (dbHook) Levels() []logrus.Level {
	return []logrus.Level{
		logrus.PanicLevel,
		logrus.FatalLevel,
		logrus.ErrorLevel,
		logrus.WarnLevel,
		logrus.InfoLevel,
		logrus.DebugLevel,
	}
}

func (db dbHook) Fire(e *logrus.Entry) error {
	_, err := db.Exec("insert into log (time, level, message) values (?, ?, ?)",
		e.Time,
		e.Level,
		e.Message,
	)

	return err
}

func main() {
	db, err := sql.Open("sqlite3", "log.db")
	if err != nil {
		log.Fatal(err)
	}

	_, err = db.Exec("create table log (time, level, message)")
	if err != nil {
		log.Fatal(err)
	}

	logrus.AddHook(dbHook{db})

	for i := 0; i < 10; i++ {
		logrus.WithFields(logrus.Fields{
			"animal": "walrus",
			"number": 1,
			"size":   10,
		}).Info("A walrus appears")
		time.Sleep(time.Second)
	}
}

It is an incomplete solution to your original problem because:

• it uses sqlite instead of mysql because I don’t have a handy mysql to use
• entry.Data is ignored. Proper handling depends on your situation and requirements.

However, it is enough to find the structure:

• Create a type that implements logrus.Hook, in this example dbHook
• Setup and instance of that type. In this case I opened the database and created a table for the logs
• Register the hook with logrus using logrus.AddHook

This structure can be adapted to fit mysql, redis, or any other storage mechanism you need.

[Nirmal_Singh]

Thank you so much for this.

Can you also tell me how to move error logs data from redis to Mysql server?

[nathankerr]

Sure.

Write a program that reads the logs you have in redis and writes them to mysql. The details depend on how the logs are stored in redis and how you want them stored in mysql.

[Nirmal_Singh]

Thanks.
I have stored the logs in redis in json format with time stamp keys and now want to save These json data to mysql in different columns after parsing the json logs data.

[nathankerr]

1. read log from redis
2. unmarshal json to a struct (use encoding/json)
3. write log to mysql

I can’t give a more specific answer because I don’t have more specific details of the problem you are encountering. Only you can get the specifics. It is your data in your systems with your requirements.

Start writing code, reading docs, and searching for specific answers to specific problems you have. When you can’t find a suitable solution, then ask specific questions. It is especially helpful to include code as part of your question.

[Nirmal_Singh]

Thanks you so much.