Understanding the memprofile output behavior

Hello,

I am not sure how to interpret the following memory profile output. I think I am fundamentally misunderstanding something about it.

The output shown is gotten using the “-base” option when viewing profiles. (inuse_space)
//note : The garbage collector is forced to run before taking a memory profile.

The scenario

  1. Start up the service
  2. Take an initial memory profile
  3. Join a device. (Create a goroutine)
  4. Configure the device
  5. Leave the device (exits the goroutine)
  6. Take a post-operation memory profile.

After investigation, I see the goroutine that handled the device has a cumulative allocated size of 15kb.

Since the goroutine has been exited, this means that the data allocated (in this routine) on the heap must still be referenced somewhere. Otherwise, the garbage collector would have cleared it. (Looking for confirmation on this way of thinking).

Now taking a look at the lowest level of my code that was called by the goroutine, I see 14.62kb cumulative allocated by the json.Marshall function.

.    14.62kB           				kvPairExportJsonResult, err := json.Marshal(kvPairExportResult) 

This means that somewhere down the line, the json.Marshal() function stores some information on the heap that is still referenced after, the calling routine calling has been closed. (Looking for confirmation here to.)

  1. I assume the underlying code of the json.Marshal() won’t keep references to data it parses.
  2. So I am forced to assume that “kvPairExportJsonResult” is being referenced my some part of my code.

Looking at the code I couldn’t immediately find something that would create a reference to the kvPairExportJsonResult. After which I decided to some testing.

//adding test1 variable with the same call
//since test1 is never used, only printed. I expect that test1 is not maintained on the heap after the goroutine finishes.

kvPairExportJsonResult, err := json.Marshal(kvPairExportResult)
test1, err1 := json.Marshal(kvPairExportResult)
fmt.Println(test1, err1)

//Looking and the memprofile output, the test indeed meets my expectations.
.    14.62kB          kvPairExportJsonResult, err := json.Marshal(kvPairExportResult)
.          .          test1, err1 := json.Marshal(kvPairExportResult) 

The real confusion starts when I turned this around.

//Expect test1 to not be allocated on the heap since it's never referenced further (only printed once) and the goroutine exits.

test1, err1 := json.Marshal(kvPairExportResult)
fmt.Println(test1, err1)
kvPairExportJsonResult, err := json.Marshal(kvPairExportResult)

//Looking at the memprofile output, it is clear that I am not seeing what I expect.
.    14.62kB	    test1, err1 := json.Marshal(kvPairExportResult) 
.          .        fmt.Println(test1, err1) 
.          .        kvPairExportJsonResult, err := json.Marshal(kvPairExportResult) 

Huh? Strange… Maybe because of compiler optimizations.
So I turned it off and tested again with an extra variable again.

//Now the memory should be allocated in the second call for sure....(?!?)
var test []KVPairExport
test1, err1 := json.Marshal(test)
fmt.Println(test1, err1)
kvPairExportJsonResult, err := json.Marshal(kvPairExportResult)

.    14.62kB          var test []KVPairExport 
.          .          test1, err1 := json.Marshal(test) 
.          .          fmt.Println(test1, err1) 
.          .          kvPairExportJsonResult, err := json.Marshal(kvPairExportResult)

I do not understand the behavior shown in the last 2 examples.
It even invalidates both my assumptions, and make me question my understanding on how these things work.

How can a variable that’s never used after a print still be used as a reference somewhere ?
How can a variable actually have “cumulative” but not a flat allocation? Hell why does it still have the exact amount as before? (14.62kb)
Why isn’t the last line in the last example showing allocation but the very first (and second example) is ?

I’m a bit lost with these results, I’m not sure If I can trust em.
I gladly accept any help.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.