Too much memory usage. Appreccite any help [SOLVED]

Hey Dave, you mean using go tool trace? I’m also interested how can i see a memory leak generated by a specific piece of code but the result of trace in browser seems to be only a statistics of resources on a period of time :confused:

1 Like

George, I did it and what I saw is after every db.QueryRow() memory usage increases but I couldn’t find any method de close that row… Is there any way to close it?

Tnx.

Yup, the trace tool. If you record a profile of the entire operation of your program you should be able to see an increase in the heap, or number of goroutines, over time.

1 Like

the Row.Scan function calls rows.Close() internally: https://golang.org/src/database/sql/sql.go#L1891

So as long as you’re actually calling Scan on the value returned by QueryRow, it should close the result set.

1 Like

@leogazio, i didn’t used the db.QueryRow but db.Query and not in a loop. try using db.Query instead and see if you obtain the same behaviour. also you can try (as @infogulch said that db.QueryRaw close the result set) to insert a short delay after the db.QueryRow to wait for the command to complete.

@dfc, thanks. i will consider studying the tracer.

1 Like

But I can’t do it, can’t put delay inside that loop, because that receives lots of UDP datagrams, if I put a 1 second delay and a hundred clients send datagrams at the same time, the last one will have to wait about 100 secs, this way that specific client won’t receive its ACK, and will start work badly…

Yeah I saw it now in the sql.go code you gave me but, why most of the examples on the web use defer rows.Close()? If it closes byh itself…

yeap, i think is not realy a good ideea to handle the datagrams on the fly
because you need to be quick. i think that a better approach is serializing
the requests in messages queues and handle them afterward because db
operations are slower than receiving packets and also can have errors.

You misread.

When you call DB.QueryRow(), calling Scan() on the result will automatically close it.

When you call DB.Query, you must call Close() yourself. This is often done with defer. For defer to work the function in which it appears must return.

Both are using the same method “Scan”, that’s what I said, If QueryRow automatically closes it, why Query not

Not exactly. Their usage is similar but they’re two different functions.:

  • Row.Scan. (source linked above) *Row is returned by DB.QueryRow. Its’ implementation of Scan calls Close.
  • Rows.Scan. *Rows is returned by DB.Query. Its’ implementation of Scan does not automatically call Close.
1 Like

@leogazio anyway now you use DB.QueryRow which like @infogulch says close the raw but from some unknown reason this didn’t happen. from here:

But if for some reason you exit that loop – an early return, or so on – then the rows doesn’t get closed, and the connection remains open. (It is auto-closed if rows.Next() returns false due to an error, though). This is an easy way to run out of resources.

I still believe that DB.QueryRow didn’t have the time to close the raw because another query is too quick executed in the loop until generate a memory leak :confused:

1 Like

@geosoft1

You mean it’s a bug on database/sql?

Definetely I found no way to fix it, unfortunately… What I really don’t understand is, what is GO’s Garbage Collect for, if it never works? Garbage Collector should be for these cases. Too bad… Too bad…

i don’t know if this is a bug but i guess, like i says previous, this can work if you change the approach and find a way to pool datagrams and convert them into a message queue (let say in a go routine,also see channels). next, you can handle the messages in the main loop and do the slow things like db operations.

Naaa definetely that’s not the problem… The problem is that GO’s Garbage Collector does not work, that’s a fact! I’m about a WEEK just trying to resolv a memory leak problem? Why we don’t have this kind of problem in other languages like, PHP, Python, Java, Delphi and C++? I’m really really, so frustrated, I wrote my whole application using this language and when I finally get it, it keeps increasing memory usage without any nexus? That’s really not good… Frustrating…

The problem is that GO’s Garbage Collector does not work, that’s a fact!

I’m sorry you are frustrated, but at this point in Go’s development cycle it is very unlikely that your code has found a bug in Go’s garbage collector. Sorry, the bug is in your code.

Is your code online somewhere? Real working code samples are very good for resolving issues like this.

My code is at the first message in this topic, if it’s my code, then show me… I modified that whole code, took off the infinite loop EVERY declaration and short declaration, I’m closing all the prepare statements, I’m using database/sql db.QueryRow which closes by itself. SO where my error then?

Thank you. Sorry I didn’t spot this, it’s a long thread.

Next question, how are you determining the memory usage of your application?

Can you please run it with

GODEBUG=gctrace=1

And post the full output. This will show the operation of the garbage collector and confirm if your program is leaking memory.

Smth like;

go path/to/my/program GODEBUG=gctrace=1

?