"Too many open files" - Opened files not getting closed with 'defer file.Close()'

[gnfnwgid (Gary S)]

I have a simple program that at one point reads a file for it’s contents, does something, and at the end writes some data to a file; then it sleeps for 60 seconds and repeats. I set it up so the first file usage (read file1’s content) fatally crashes the program if it fails since the code below it depends on it, but the second file usage (writing to file2) doesn’t, and only logs the error to syslog.

After a while of the program running, in this case maybe 12 hours, the program logged an error due to not being able to write data (Too many open files), then 60 seconds later (next file access attempt) logged a fatal crash for the same reason.

I checked lsof | grep programname and it keeps showing more and more open ‘connections’ to the two files. I must be doing something wrong and the files aren’t closing after they’re done being used.

Here is part 1 where file1 is getting read

data, err := os.Open(path)
if err != nil {
fmt.Print("File doesn't exist/error.")
log.Print(err)
os.Exit(1)
}
defer data.Close()

Here is part 2 at the end when it’s writing to the

file, err := os.Create(path2)
if err != nil {
log.Print("Error writing to file")
}
defer file.Close()

fmt.Fprintf(file, "%v", output)

I thought it might have been caused by writing to the output file after calling defer file.Close() in part 2 instead of before (I think it might be wrong this way) but lsof is showing the files to stay open for both files 1 and 2.

[NobbZ (Norbert Melzer)]

Are actually leaving the function where you call defer in, or are both defers in a loop?

If the latter, than your files get never closed.

[gnfnwgid (Gary S)]

I think that must be it, so I should be doing file.Close() minus the defer than?

[NobbZ (Norbert Melzer)]

Yupp

[ChrisHines]

Or consider wrapping the logic for one iteration into a function that provides a better scope for the defer.