How to best format a multi-line log entry

I have an app that keeps a log of os/exec command executions. The first part of every entry in the log is the timestamp, then the command that was executed, then its output.

The problem is that the external command may output many lines, some of which are indented or separated by a blank line (example: a stack trace). Makes it hard to find the start of the next log entry.

I admit I haven’t done a lot of research in this area yet but I’m wondering what you do to group multiple lines in a log file so it’s still easy to scan visually. Should I indent each line with whitespace or some non-whitespace character like hyphens?

(Edit: For now, I’ve been indenting subsequent lines of the same entry with four hyphens and a space which seems to flow pretty nicely, similar to valgrind but less obnoxious. Still interested in your methods though.)

Hi,
how about prepending some corelation id to output:

[foobar] executed "ps -aux"
[foobar] root      1463  0.0  0.0 116860  1364 ?        Ss   Oct08   0:01 crond
[foobaz] some other event
[foobar] root      1495  0.0  0.4 287664 16372 ?        Sl   Oct08   0:40 /usr/bin/docker -d

Of course, you would have much more ‘tags’ in logs: like timestamp, pid etc.

When app is under loads, its useful to provide 2 important values: how many ms or us passed since beginning of operation and from last log entry - it helps to measure performance.

I would read this 2 resources, very helpful:


http://research.google.com/pubs/pub36356.html

3 Likes

If debugging a whole load of statements which might take multiple lines each, I sometimes put a unique separator in, -------token------\n, so that it’s clear where one entry starts and the other ends. That might work as an alternative to indents.

1 Like

Ooo, I like the idea of a unique ID for each entry - even though they will all be grouped together in my case. Thanks for the links - gonna check those out too.

I gathered my experiences in short blog post: http://slawosz.github.io/2015/10/14/logging-best-practices/
Hope it will help!

1 Like

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.