How to store logs just before crashing

Hi,

I’m working on an application where I need to properly handle errors to avoid crashes. To help debug and resolve any issues that may arise, I want to store the stack trace of any errors that occur in a log file. I tried adding a defer function in my main function to retrieve the stack trace, but I wasn’t successful. I’ve included a sample code snippet that results stackoverflow error due to an unhandled panic. How can I modify this code to store the stack trace of the error in a log file?"

package main

import (
	"io"
	"runtime"

	"github.com/rs/zerolog"
	"gopkg.in/natefinch/lumberjack.v2"
)

func main() {
	ljLogger := &lumberjack.Logger{
		Filename:   "/var/log/myapp.log",
		MaxSize:    10, // Max size of each log file in MB
		MaxBackups: 5,  // Max number of old log files to keep
		MaxAge:     7,  // Max number of days to keep log files
	}

	// Create a multi-writer that writes to both the file and the buffer
	mw := io.MultiWriter(ljLogger)
	logger := zerolog.New(mw).With().Timestamp().Logger()
	defer func() {
		if r := recover(); r != nil {
			// Log the error message and stack trace
			stackTrace := make([]byte, 4096)
			length := runtime.Stack(stackTrace, false)
			logger.Error().Interface("panic_value", r).Bytes("stacktrace", stackTrace[:length]).Msg("Unhandled panic occurred")
		}
	}()

	abcd()
}

func abcd() {
	defer func() {
		if r := recover(); r != nil {
			abcd()
		}
	}()
	panic("something went wrong")
}

Can anyone help?

I think the most reliable way to handle that is by your supervisor/init system/system logger, to collect and write all your programs output to a centralised logfile.


Your problem here is:

You panic in abcd(), which will try to recover() itself by calling abcd(), this will cause an infinite recursion, eventually blowing your stack away and resulting in an error that you can not recover from, as the Go-runtime hard-kills your program on a stack overflow without giving you the opportunity to recover from it.

In my opinion, a centralised handler that logs panics is acceptable, though it shouldn’t rely on the logger (it could be what has crashed) but only on “primitive” operations and it also should not rely on anything but stdout/err, as that might not be available anymore. You never know what crashed.

Besides of this centralised panic handler to get some log out, you really shouldn’t recover() at all! Use any error return you get and if some function you call uses to panic while not returning an error, change it to do proper error handling or create a bug ticket upstream!

My real problem:

I have created an MSI package from golang webapp. But this app crashes due to unknown error and I am not able to see or reproduce that issue. I know that it is due to unhandled error. But I could find the exact location, that was causing the issue. to fix or handle that error, I need to see the stacktrace.

In the above code, I called abcd() recursively to just crash the execution.

As I have said, calling abcd() recursively from the first error handler will just blow your stack, that is nothing you can recover from.

I trimmed down your example code a bit, and the linked playground works for me and prints the trace to the terminal, as well as the length (I added that to verify it is indeed the custom handler rather than the default behaviour).

Though instead of pre-initialising a buffer with len 4k, I’d probably prefer one with a len of 0 and a capacity of 4k (make([]byte, 0, 4096)).

Also, by adding r to the output, you can add the original message, as you can see in Go Playground - The Go Programming Language

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.