I’m curious about the current state of garbage collection for slices in Go. Is the Go garbage collector still unable to collect objects that are no longer accessible but are referenced by the underlying array of a slice?
For instance, in the following code, is it possible for the garbage collector to collect a at line 5?
a := 1
s := make([]*int, 2)
s[0] = &a
s = s[1:]
// GC at this point
Are there any tools to detect this? I checked Staticcheck and go vet and it seemed that none of them can detect the issue.
No, you’re actually referencing the memory of the underlying array, so GC won’t reclaim this part of the memory.
You can use the functions of the runtime package to trigger and view the memory usage in imitation of active GC.
Thanks. But what I pay attention to is the a object, which is referenced by inaccessible elements s[0] of slice s.
If I use s[0] = nil before assignment s = s[1:], a can be reclaimed by GC at the last line, right? But if a programmer forgets to write s[0] = nil and the GC is unable to recognize the inaccessible a and reclaim it, much memory may not be reclaimed.
I think you’re oversimplifying and don’t know if you know about the unsafe package, which can be similar to the offset point of C (this is just an example), and simply reclaiming memory will cause unexpected problems.
s -> s[0] -> a
s[1]
// s=s[1:]
s[0] -> a
s -> s[1]
When you use s, you’re actually referencing the entire underlying array, and GC doesn’t recycle the memory of the underlying array. At the same time, because the underlying array references the memory that A points to, the memory of A will not gc.
The memory pointed to by a will only be freed if there is no reference to the underlying array in your context. It’s not hard to see why.
As for why GC doesn’t handle this kind of thing for you, I think you’re putting the cart before the horse: the ease of use of Golang doesn’t mean it stops you from writing bad code at will.
Awareness of memory allocation is a basic quality for developers (I admit that golang is easy to get started with, but I’ve encountered a lot of developers who write bad code).
Thanks for your detailed reply. I know the unsafe package and its possible effect to GC(I’m not so sure, so I’m asking here). What about set nil to s[0] (manually or semi-automatically) to make a not referenced by the underlying array of s? I think it’s a good practice if a can be reclaimed immediately instead of being reclaimed at the same time of reclaiming underlying array of s, i.e. shortening the lifespan of a to its actual lifespan.
As the above reply mentions, can GC handle nil smartly? Is that true? Is there any source code or blog that I can reference? Or somewhere else to negate the predicate “GC is smart enough to handle such cases on its own”?
New generic slices package has Delete function. It zero/nil out the obsolete elements, so they can be collected by GC. But at the same time underline array will not change its capacity. Thus, the memory already allocated for it will stay the same.
I don’t think you understand what I’m talking about.
As I said above, it’s essentially because the underlying array of s has a pointer to A, so A can’t be reclaimed by GC memory, which is the simplest and clearest way to put it. So if the underlying array of s does not have a pointer to A, then A will be reclaimed by the GC memory, note that this is not an immediate collection, the GC has its own set of logic.
If you don’t set nil and S is used all the time, then A memory will never be reclaimed. (On this note, you can write a runtime to see the memory footprint of the sample code, it might be more intuitive to turn a into a long string)
var m runtime.MemStats
var a string
a = uuid.NewIdn(1024 * 1024)
fmt.Println(len(a)) //len 1024 * 1024
s := make([]*string, 2)
s[0] = &a
for i := 0; i < 10; i++ {
time.Sleep(100 * time.Millisecond)
runtime.ReadMemStats(&m)
fmt.Println(m.Alloc)
runtime.GC()
}
s[0] = nil //Please comment on this line repeatedly for comparison
s = s[1:] //Please comment on this line repeatedly for comparison
for i := 0; i < 10; i++ {
time.Sleep(100 * time.Millisecond)
runtime.ReadMemStats(&m)
fmt.Println(m.Alloc)
runtime.GC()
}
fmt.Println(s[0]) //Please comment on this line repeatedly for comparison
As for automation, no! I repeat, no!
Golang’s GC doesn’t automatically set up nil for you, and if it does, it will lead to a host of other problems, so the best thing to do is to do nothing and let the developers solve the problem themselves.
Sorry for the ambiguity caused by my last reply. I try to write a linter to provide a hint for the programmer to set nil if it won’t cause wrong behavior of the program. “immediate collection” is not what I want and I know it’s hard to achieve. But speeding up the collection of A may help reduce the size of memory.
For “automation”, I mean using static analysis tools to provide some advices because programmers may forget to set nil. It’s nothing to do with GC. Sorry for the ambiguity.
Unfortunately, golang can figure out for itself if the memory needs to be freed. Most of the time, setting nil is the same as not setting nil, depending on your context.
As for static analysis tools, it’s a nice idea, and I would try to use one if it were available, but it’s complicated. If the context is not carefully analyzed, it can be annoying if the consumer is misled to actively set nil on the object being used.
Good luck.
So, I write a new example and I believe it shows the effect. (I should have shown the program earlier but I do not know that I can use runtime.ReadMemStats and runtime.GC to show the effect)
type LargeT struct {
arr [100000]int
}
func main() {
s := f()
var m runtime.MemStats
for i := 0; i < 10; i++ {
time.Sleep(100 * time.Millisecond)
runtime.ReadMemStats(&m)
fmt.Println(m.Alloc)
runtime.GC()
}
for i, elem := range s {
elem.arr[0] = i
}
}
func f() []*LargeT {
len := 1000
s := make([]*LargeT, len)
for i := 0; i < len; i++ {
ptr := new(LargeT)
ptr.arr[0] = i
s[i] = ptr
}
start := len - 200
// comment the for loop for comparison
for i := 0; i < start; i++ {
s[i] = nil
}
s = s[start:]
return s
}
use go 1.22.0 to execute the program. The results with/without setting nil are shown as follows:
Although the example is a synthetic example, the pattern exists. And now I know the current state of garbage collection of slice through ReadMemStats API. Thanks.
Besides, thank you for reminding me that static analysis tools can be annoying and it’s really a challenge.
The reason why I use string is that the int type occupies a very small size and cannot be seen. Using string to detect memory usage is a good choice.
uuid.NewIdn is my own code function. It is normal that you cannot find it, but I also marked it from the comments that this is a string function that generates a specific length and is used to generate test data.
Multiple runtime.GC is to actively trigger asynchronous gc (simply calling it once will not trigger it immediately.)
As you can see, there are certain harsh conditions for actively setting nil. It is difficult to know whether nil needs to be set from the code snippet. When misleading, the user uses nil data, which can easily cause panic.
For example, golang usually leaves it to the user to decide on uncertain memory, and does nothing by itself.
(Sometimes some static analysis tools are a bit annoying. For example, defer conn.Close() always prompts me to handle errors, which is very annoying; but it prompts me that context.CancelFunc is not used, which makes me satisfied. )
)