Too much memory usage. Appreccite any help [SOLVED]

Getting Help
41

Nope, at the front, it’s an environment variable.

42

I know what’s an environment variable, I didn’t know this one is… But thank you anyway, I’m gonna check it out…

43

Just one doubt, I’m using this odbc library;

This library got so many lines of C code. Could it be the problem? Maybe…

44

Here’s an example:

GODEBUG=gctrace=1 godoc -http=:8080
45

Ok and what do I do with that env var? Where do I use it? How do I use it? I search whole Google and find no consistent information on how to do it…

46

Please try setting the environment variable, running your program, then post the complete output when your program gets to the point that you feel that it has leaked significant memory.

47

…? My program is a service application with hundreds of lines inside an infinite loop… Where do I see this? What is that 8080 http param for? A HTTP server? I’m sorry but this explanation is incomplete and does not help!

48

Please try this command

GODEBUG=gctrace=1 ./yourprogram

Replace ./yourprogram with the correct command to run your program, then post the complete output when your program gets to the point that you feel that it has leaked significant memory.

49

godoc is Go’s version of man(1)
-http=:8080 causes godoc to run a web server rather than output to a console.

This was just an example that you could run to understand how setting GODEBUG=gctrace=1 will affect the output reported by a Go program.

http://go-talks.appspot.com/github.com/davecheney/presentations/seven.slide#10

1 Like
50

@dfc, seems that he isolated the memory leak around db.QueryRow but, anyway, tracing is good idea…

@leogazio, just an idea… instead calling db.QueryRow try to make a wrapper function who call db.Query with a defer rows.Close().

1 Like
51

@geosoft1 I did it and nothing changed…

52

Youcan try https://github.com/alexbrainman/odbc - the one you linked in uses too much runtime.SetFinalizer for my taste…

1 Like
53

I just tried that one too and nothing has changed… I tried all SQL Server and ODBC drivers but no one has changed this memory leak…

54

Then it is your code.

Can you provide a nice, formayted, refactored, fixed - your current version of code?

55

Current version is that one.

56

Sorry, but I see only Too much memory usage. Appreccite any help [SOLVED] from 4 days ago.
That code still declares variables waay before use, uses unneeded helper functions (funcoes_gerais.CopyStr, funcoes_gerais.OutOfRange), uses defer to closed prepared statement in for cycle (effectively not closing the prepared statement), an that main code is still a long spaghetti of code.

57

I did everything everyone told me here to do, and nothing has changed, my code became a salad, after those changes and still hasn’t solved my problem. I’m looking for consistent information on how garbage collector works and WHY it does not turn my memory back, if it exists to do that work… I really wanna understand…

58

The defer in the for cycle alone is a big red flag - the deferred code runs only at the end of the main function, so in every cycle you create and prepare a new statement, effectively never freeing them, building up memory.

Without fixing such easy flaws, we don’t see the remaining, more hidden ones!

59

I already changed it, took all defer rows.Close() and replaced them to rows.Close() at the end of the execution, nothing changed… I did all you guys can think a human being could do, definetely it’s not my code, for sure! I need to understand how garbage collector in GO works, that’s not normal…

60

So, you can’t share that changed code?

Have you tried profiling your program? net/http/pprof allows profiling running program: just

import _ "net/http/pprof"
func init() {
  go func() {
	log.Println(http.ListenAndServe("localhost:6666", nil))
  }()
}

and
go tool pprof http://localhost:6060/debug/pprof/heap
top20 or web will show the code most allocating.

1 Like