[important] How much memory will an int32 var actually consume?

JOhn_Stuart · September 24, 2019, 6:21am

Hi all,

Here is a tricky question that is puzzlng me:

i := int32(3)

This piece of code will declare a variable, inside a function. Let us assume this variable will escape to the heap, thus needs to be allocated on the heap. Just for a sanity check, I believe it will escape on the heap if , after declaring it, we can find at least one of these 2 situations:
j := &i
or:
var j interface{}
j = i

Now, hear me out, my i variable of type int32 is declared on the heap. This means that the compiler will tranparently replace my i with a pointer, thus allocate a int32 variable on the heap, and also a pointer. Please correct me on this.

So far we have spent one int32 = 32 bits and one pointer = 32bits on a 32 bits processor machine.

I also expect variables declared on heap probably have an extra field declaring their type. Is this true ? If so, another 32 bits (maybe less) is being spent ?

In conclusion, my questions regarding an int32 allocated on heap are:

how can I test/see how much memory is being allocated when executing a given line of code ?
how does the Go runtime know that my int32 is actually an int32 without allocating an extra field to it ?
will it actually allocate an extra field to it that indicates its type ?
will Go runtime also maybe add another extra field that acts as a lock, similar to Java ?
will there be any onther memory allocated, on top of the questions above, for my simple int32 value ?

Please point me to tutorials, I am sure someone has already investigated such an interesting topic.

lutzhorn · September 24, 2019, 6:59am

Why is this important?

JOhn_Stuart · September 24, 2019, 7:33am

It is the ultimate interview question.
It marks my passing from middle level to senior go developer.
I can better think and understand what is going on with my code.
It can potentially save the planet by making my code use less resources.

I cannot stress the last one enough: for those who believe in climate change, the environment revolution will come from engineers that find practical methods to reduce waste.

Plus it shows an advanced level of technical expertise, if one is able to understand the inner workings of a programming language. Maybe subjects like this would inspire young developers to choose Golang as their main language.

hollowaykeanho · September 24, 2019, 8:25am

Not enough information to conclude. You need disassemble it to check the status of stack/heap.

Taking that assumption that i is definitely declared on heap, i is still retaining its int32 data type nature, which is a valid int32 variable storing 3 as value. j however, holds the memory address of i instead.

Excluding j, yes. If j is initialized using j := &i, then j itself is a valid variable holding pointer value (32-bit memory in your case). If j is included, it should be 2 int32 data size, 2 32-bits pointers.

However, if j is initialized using:

var j interface{}
j = i

It is very hard to tell unless you disassemble it and analyze from there. interface{} is not an object and it has special meaning in Go. Long story short, you should not treat interface{} as a generic data type like the one in Java, although it behaves almost like one.

2 ways you can do it:

You build a benchmark performance test to output specifically the memory performance.
You use unsafe package to check it, on the run.

For such experimentation, I will use the latter option since it’s easier. Here is an example:

package main

import (
	"fmt"
	"unsafe"
)

func main() {
	i8 := int8(3)
	fmt.Printf("Type:%T | Address:%v | Pointer Size:%v | Mem Size:%v\n",
		i8,
		&i8,
		unsafe.Sizeof(&i8),
		unsafe.Sizeof(i8))

	i16 := int16(3)
	fmt.Printf("Type:%T | Address:%v | Pointer Size:%v | Mem Size:%v\n",
		i16,
		&i16,
		unsafe.Sizeof(&i16),
		unsafe.Sizeof(i16))

	i32 := int32(3)
	fmt.Printf("Type:%T | Address:%v | Pointer Size:%v | Mem Size:%v\n",
		i32,
		&i32,
		unsafe.Sizeof(&i32),
		unsafe.Sizeof(i32))

	i64 := int64(3)
	fmt.Printf("Type:%T | Address:%v | Pointer Size:%v | Mem Size:%v\n",
		i64,
		&i64,
		unsafe.Sizeof(&i64),
		unsafe.Sizeof(i64))

	i := int(3)
	fmt.Printf("Type:%T | Address:%v | Pointer Size:%v | Mem Size:%v\n",
		i,
		&i,
		unsafe.Sizeof(&i),
		unsafe.Sizeof(i))
}

// Output:
// Type:int8 | Address:0xc000096000 | Pointer Size:8 | Mem Size:1
// Type:int16 | Address:0xc000096020 | Pointer Size:8 | Mem Size:2
// Type:int32 | Address:0xc000096024 | Pointer Size:8 | Mem Size:4
// Type:int64 | Address:0xc000096030 | Pointer Size:8 | Mem Size:8
// Type:int | Address:0xc000014160 | Pointer Size:8 | Mem Size:8

It’s defined in your code, as in data- type declaration at variable creations. If you define the variable with a value, the default data-type for that value is used.

Strictly speaking, no. Data types should stop at compiler level except when you explicitly compile the binary with debugging symbols. It is only useful for human-side, not machine side.

No it makes no sense because not all programs use concurrency. In those situation, creating additional and unused data fields is a waste of resources.

Yes being the reason: you created either a mutex lock or a channel, assuming you’re referring to the Java’s extra lock field.

Take it as a grain of salt…

p/s: thanks for asking.

Let’s amp it up a bit…

You have to Google on your side for read-up resources instead of asking others to do it for you. This helps you in your career by asking proper questions and exhibits some level of professionalism.
You’re equipped with disassembler knowledge with your previous question. Do try them out on your side so you can trim down self-answerable questions like the one I mentioned above.

Do note that some powerful developers (of I worked before) exhibits weird and interesting social behavior. This statement is very offensive to them and some do hurl sharp fruit knife across the desk and detests you for life as if you cheated his/her +1. It will be life saving tip.

I’m not sure are you aware that you’re either digging into the deep rabbit hole of unknown or a hole for yourself in this case. If you read through all the materials and its linked expansions asked in your other threads, you should realized by now, in Go, we do not need to manage stack/heap explicitly or manually unless you’re TinyGo developer, which the project is still developing the no-GC MMU. Your interviewer is likely look into something else that is not technical.

I’m sorry to hear that you exhibited what we called “premature optimization”, if you understand my 3rd point.

Just these 4 points are good enough to show that you are not ready to be a real senior developer yet. Those are junior developers (of any languages) trait and attitudes. You might want to work on them.

Best wishes in your career advancement.

Reference

JOhn_Stuart · September 24, 2019, 12:30pm

Hi @hollowaykeanho & @lutzhorn,

Thank you for your replies so far, they are a step forward in the right direction.

I have used unsafe to check the size of an int32, on my machine:
Type:int32 | Address:0xc00009e000 | Pointer Size:8 | Mem Size:4

It seems almost impossible, for me, that the go runtime allocates just 4 bytes for an int32. In java it is almost 3 times as much, due to various hidden fields that are also transparently allocated. I mean how would even one implement a Go runtime, without extra (hidden) fields to all objects ?

I intend to figure out exactly how much memory will this line allocate:
i := int32(3)

So far, my next and only 2 leads are the benchmarking and the disassmbling methods. I will keep you updated on my findings.

Please feel free to point out tutorials or discussions on this topic, as alone I was unable to find any. Java has multiple such tutorials, including diagrams with the extra hidden fields for any library data type.

I hope Go has similar explanations & diagrams and if not, I will do my best to post any conclusions in this thread.

hollowaykeanho · September 24, 2019, 1:07pm

Take a moment and analyze the output data. It’s data. 32-bit is 4 times of 8-bit, why would you be surprised that it allocates 4 bytes memory space when there is no underlying any extras? In C/C++/embedded-C, it is very common. We even have the habit to count memory (called “memory counting & tracking”)

Because Java is an entirely different tool. Java uses JVM, in which Go does not. Go is somewhat similar to C/C++, compiled to machine language, specific to OS. Similarly, Cython, Jython, and Ruby consumes larger memory because of their script interpreter layer.

What you just experienced is one of the many powerful reasons to use Go. It’s time to let go the past, stop force-fitting other tools into Go, and absorb Go like how you did for your first programming language.

A sledge hammer cannot be forced to perform rivet gun’s tasks, it will turn out very ugly.

hollowaykeanho · September 24, 2019, 1:10pm

You ignored salt point No. 2 again. Try to understand it, play with it, let the knowledge flows into you.

You can’t hack your ways to become a senior developer. It’s a career suicide direction.

HINT: you already answered your question.

system · December 23, 2019, 1:19pm

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.