I am self thought golang user, otherwise mechanical engineer (so far from a programming expert). I switched from python to golang to speed up my project code as it can run for days, so any speed up is really appreciated, especially as I use the code for analyses on daily basis.
I have been running regularly pprof profiling and optimized use of libraries wherever possible or replaced it with own code. So far I am pretty happy, got the speed up wrt python for 82x, which is awesome. I believe my code is pretty optimized as pprof tool shows that my functions are not at the top anymore. At the moment, about 56% of the time is used by the functions runtime.mapaccess1_faststr and runtime.mapaccess2_faststr, which as indicates is accessing parameters in my map[string]map[string]*SomeParams. Is there anyway to improve or workaround this part or have I hit more or less the bottom? Thank you kindly for any expert opinion or a hint on this part.
Are the string keys to these maps a known set (that is, are there, for example, 8 possible keys in either the outer or inner map(s))? Maybe you could use a struct instead?
Can you pair together the keys, so instead of map[string]map[string]*SomeParams, where you change your access the values from m["key1"]["key2"] to m[[2]string{"key1", "key2}]map[[2]string]*SomeParams? That should make it a single map access instead of two. But it’s still hashing/comparing two strings, so it might not actually help.
To give any other ideas, I would have to see the code to see what you’re doing with the map, specifically.
Hi @skillian , thank you for your email and feedback, appreciated.
Your questions are all related to the code, naturally, so I’ll give a glimpse on it as the code is in overall +30k lines.
map[string]map[string]*SomeParams
In general, the code is very simple, the first keys of the map are simply dates, and the second keys are a system components, its process controllers etc. and then *SomeParams are components/controllers related parameters, measurement values etc. For Example:
So, the first keys (dates) are in magnitude of hundreds (can be thousands), the second keys (components/controllers) in thousands and *SomeParams in hundreds as well. *SomeParams are most of the time engineering equations like heat and mass balance, Fourier-analysis parameters, etc. This is basically whole philosophy, which ends up on a huge number of map calls. Any ideas on improvement or reasonable restructuring?
Thank you very much.
PS: Thank you for the suggestion: m[[2]string{"key1", "key2}]map[[2]string]*SomeParams though in this case am not sure if this would be OK from the clarity of code stand point, perhaps speed too as mentioned.