What would you think about a language construct that enforced stack allocation and "locality"/non-escaping?

Technical Discussion
1

When optimizing a Go application, one way is, with help from the debugger, to reduce the number of heap allocations. This is somewhat of a guess work, and it’s also not explicit when reading the code if a variable will be stack or heap allocated. I had an idea of an experimental language feature that could remedy this.

Example:

// v1 is either stack or heap allocated depending on context
v1 := Point{1, 2}
// v2 is always stack allocated, but is NOT allowed to escape
local v2 := Point{1, 2}

The crucial point is “not allowed to escape”. Go is already using escape analysis to decide what can be stack allocated. It’s a common optimization technique in many languages. Adding local would throw an error during compile-time if a variable was detected to escape (in principle, if the variable or its memory area can still be referenced when the current scope disappears).

This feature is kind of similar to what they’re doing in Rust with lifetimes. Lifetimes are more powerful but also puts a greater burden on the programmer. Non-escaping variables would give the programmer the possibility to “opt out” of garbage collection in a predictable manner, in tight spots where optimal performance is required, instead of playing Whac-a-mole with the debugger.

Non-escaping is not the same as non-aliasing, btw. Compare with the uniqueness type.

Non-escaping variables can still be passed down the stack chain using references, but not up (can’t be returned without a copy (this copy might be optimized away by the compiler, as done in C)).

There are other optimization techniques related to escape analysis that are not considered by this idea, e.g. scalar replacement.

Another way to opt out of GC are value types in C# (not yet added to Java). It’s somewhat ad hoc. Value types are implicitly copied, as they are in Rust and C (some of those copy instructions can be optimized by the compiler).

One possible prototype implementation would be to make a Go parser that compiles to Go, but adding the new keyword (removed after compiled) together with the escape analysis on the abstract syntax tree.

Let me know what you think. :slight_smile:

Olle

2

I am sorry to disagree, but I don’t care for a language feature whose purpose is solely to facilitate optimization.

I suspect you know this, but values escape to the heap if the compiler cannot prove that their lifetimes are bound by the lifetime of the function they live in, or if they’re deemed too big to live on the stack. If you ever pass a value to an interface function, that value will escape to the heap, e.g.:

// MyReadAll is a poor (potentially disfunctional) implementation of Go's io/ioutil.ReadAll.
// It doesn't matter for my example.
func MyReadAll(r io.Reader) (bs []byte, err error) {
    bs = make([]byte, 4096)
    var total, next int
    for {
        if next, err = r.Read(bs[total:cap(bs)]); err != nil {
            break
        }
        bs = append(bs[:cap(bs)], 0)  // go default capacity increase
        bs = bs[:total+next]
    }
    if err == io.EOF {
        err = nil
    }
    return
}

bs must escape to the heap because who knows what r will do with it.

If your example point escapes to the heap, I would be interested in seeing the sample code and potentially suggesting improvements to address that.

If this is something that is a constant source of issues for you, I’d recommend inventing a magic comment and writing a tool that uses the go/* packages to parse your source and the output of Go’s escape analyzer to identify when this happens.

3

It’s OK to disagree. :slight_smile: You don’t care for parallelization either? :wink:

bs must escape to the heap because who knows what r will do with it.

Well, in the case of non-escaping variables, restrictions must be put in place. Mostly that no function is allowed to escape it, but can alias it. These static restriction will make it possible to know that it won’t escape. And yeah, in another language, interfaces would have to be annotated if they escape variables or keep locality. Either that, or, as you point out, assume it escapes if it cannot prove locality (third-party code etc).

If your example point escapes to the heap, I would be interested in seeing the sample code and potentially suggesting improvements to address that.

I don’t have any example code, sorry. This is just a thought experiment.

4

What makes you say that?

I still think that adding a keyword to the language is the wrong way to solve this problem (I’m not even sure I agree that values escaping to the heap is a problem). I feel the Go language should not provide that kind of control. Go is designed to be easy to read. It was designed with garbage collection so that programmers didn’t need to manage memory. I feel adding a keyword to control memory allocation goes against that idea.

1 Like
5

What makes you say that?

Hehe. Becaue parallelization is a language feature that exists for one reason only: optimization. :smiley:

I feel the Go language should not provide that kind of control. Go is designed to be easy to read.

Well, need is the mother of invention. :slight_smile: All languages grow. We’ll see.

It was designed with garbage collection so that programmers didn’t need to manage memory.

If devs optimize their Go programs by reducing heap memory, that’s already not the case. Here’s one article (that you might disagree with ^^ ): Golang Memory Management: Allocation Efficiency in Go Services

Of course, the need for tight memory control like this depends on which domain you work in. You might argue these people shouldn’t use Go in the first place, but rather C, C++ or Rust.

6

I’m not sure I agree with that. For example, the C++ language and memory model didn’t acknowledge that threads exist until C++11, but people were using libraries to accomplish this for years (decades?) before C++11. In the case of Go, the go keyword starts a concurrent routine (if the GOMAXPROCS environment variable = 1, then there is no parallelism but programs/libraries can still use the go keyword). Perhaps the go keyword is ultimately for performance, but I view it as a way to describe that a piece of code can run independently of the main routine flow.

True, but what’s interesting to me about Go is how the designers try not to adopt new features just because other languages have them (e.g. https://www.youtube.com/watch?v=rFejpH_tAHM&t=66s) even if the feature is useful.

What I’m questioning is the “need.” I don’t think that the solution is a new keyword and I also don’t think that the problem is that a value escapes to the heap. The problem probably starts off as some functionality was expected to perform at a certain speed and after development, that didn’t happen. After debugging, it turned out one or more values escaping to the heap slowed it all down. I don’t think the fix is to put a keyword to lock the variable down and then report an error if it has to escape, I think it could instead be accomplished with a linter rule.

I’m not against optimization. I’m against adding language features solely for optimization. If one or more variables are going to escape to the heap, I try to put them into a struct that escapes earlier up the call chain but that I can keep on re-using later, so though it results in an allocation, it doesn’t result in n allocations. It often results in uglier, less Go-like code, but if the performance is critical, I either go with the uglier code or refactor to make the problem go away (e.g. why am I making n calls to a function whose value escapes to the heap anyway?).

I’ve skimmed through the article you linked and it looks interesting. I’ll give it a more thorough read later :slight_smile:

7

Here’s an anecdote about switching from Go to Rust because of performance issues: https://blog.discord.com/why-discord-is-switching-from-go-to-rust-a190bbca2b1f

Not saying my suggestions would solve their issues, of course. And their issue with Go GC might already be patched in Go.

1 Like
8

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.