Imagine I am writing a Go package that aims to optimize something, but the user can define his or her own optimization function and another function to check conditions. These functions can differ and can have different signatures from problem to problem. At the end, the user will use a big struct that will use pointers to these two functions, but the problem is that these functions can use different arguments… For instance, one problem can use func(x, y int) float64 and func(x, y, z int) bool while another func(x []int) float64 and func(x []float64, y bool, c float64) bool. Hence, I cannot use a struct with two pointers to two functions, since I have to precisely define the functions’ signatures. Frankly, I have no idea how to do this.
I know the above description is a little vague, but I hope you know what I mean. Do you have any ideas how to attack such a problem?
For you case, currently, my approach is defining 1 “generic” data structure having only multiple parsing interface (input), and printing interface (output); with the data structure geared towards your optimization function.
Then I pass that data structure into a single optimization function as input and use pointer to update its internal data structure with output value.
This approach allows me to:
Isolate the algorithm away from any input parsing changes.
Manage parser’s and printer’s creation/deprecation easily by adjusting the data structure.
Allows me to define the optimization function interface if needed.
Allows me to optimize / re-structure without affecting the public facing interfaces (as in parse, print, and optimize function).
Thanks a lot. It does help, though I must admit my imagination, at least for the moment, does not enable me to imagine the final design of code. One of the reasons is that I am a Go newbie, so some issues constitute quite a difficulty. Are you maybe aware of some standard library packages that are build around such an idea?
You can take a look at the common part (common.go) of the TLS package (https://golang.org/src/crypto/tls/). As you know, TLS supports a large number of ciphers and some variants contain their respective specific configurations (e.g. RSA vs Elliptic).
For customer side, you can take a look at net/http package (specifically net/http/transport.gohttps://golang.org/src/net/http/transport.go) on how the tls.Config is being deployed.
TLS uses the Config data structure to keep a common input throughout its own package so that whenever there are new changes coming in, they only need to update the data structure accordingly without messing with other parts. Example, currently many is still using TLS1.2 while TLS1.3 is still under development. By the time TLS1.3 is ready, the HTTP package “customers” only updates 2 flags in the on their side (Config.MinVersion & Config.MaxVersion) and that’s it. They don’t really go deep (except security analyst and maintainer) into TLS1.3 inner working anyway.
The idea is not that hard to figure out. The whole idea is to think “architecturally” (not the right word but basically, zoom your thinking a little out):
Start from your function with a common data structure as input.
Then expand the common data structure with all the I/O variances using Go interface method.
In your case, you common data structure is likely gonna have private data variables so that you only want to expose those interface methods to the public. The flow will be:
Customer create “Data” struct and named it as a
use a.ParseInputTypeA(...) or a.ParseInputTypeB(...) or a.ParseInputTypeAV2(...). You get the idea.
run pkg.MySecretRecipe(&a) to operate your optimization [As long as you operate with pointer, you’re fine]
Like step 2, use a.PrintOutputTypeA() to dump specific output of specific version.
I suppose the idea is not as complex as it seems to me, but since I came from rather different languages (Python and R), I still need to change my thinking (e.g., in Python, achieving this would be a piece of cake). For instance, I have never worked with interfaces, and though I get the general idea, I still need to not only understand it but also feel the vibe, in order to be able to design more advanced structures.
There is one problem, though. I think that in the case of the package I mentioned, I would need to pass quite a significant part of coding the idea to the user. This is because the user needs to be able to define own functions to be used in optimization, with various conditions to be checked.
For this very reason, I cannot write the parsers you mentioned (a.ParseInputTypeA(...) or a.ParseInputTypeB(...) or a.ParseInputTypeAV2(...)), but the user will need to do it. This is because it’s not just adjusting to several possible types to be used, but to an infinite number of them. (Well, maybe not infinite, but quite a few.) For instance, I myself can immediately think of functions that use two []float64 slices and a float64, several []float64 and two []int slices, and various combinations of these situations. But the whole idea is to make the package more generic, so that the user be able to use it for a completely different set of the two functions. I will cover several common situations, but I do want to give the user an opportunity to use any function they want.
If all the above can’t handle the chaos, you might need to tweak your algorithm a little.
Haha. addicted to magic? Former Python and Ruby developer here. No worries. Yeah, need change quite a bit. Go is very C-like: detail-oriented and clear.
I know I won’t! It’s already my favorite language, though I have no idea why! Just love it, maybe for its simplicity, maybe for performance, no idea. I do miss some generics, and I know it’s to come.
I do hope to see generics soon, because for the moment this is the only thing that I miss. In quite a few situations, though, we can do good stuff without generics, by good code design. But there is situations like the one we’re discussing when Go limits us maybe too much. For the moment I write this package to learn, but eventually I do want to write it and publish. Hence I will likely wait for the generics to come into Go 2. I hope this will indeed make it simpler to design such a package.
Sure, I can actually make the whole idea less generic, by implementing some 10 or so common situations and that’s it. But this would mean that to even slightly adjust the algorithm (I am afraid, here an often situation), the user would need to change quite a lot of code. This being the case, I don’t think it would make sense to write this package in Go. I want to give the user a Go package that will be easy to use, not something that will give a lot of work (and to adjust it, the user would likely need to understand quite some part of the code of the package, something that would need rather too much time). Perhaps Go was not aimed at such situations, I understand that. But I so much want it to work in such situations! I can easily write this thing in R or Python, but I so much want to write it in Go! I can easily then make it both a package (if one wants to adjust it) and a small portable command-line program to run the optimization for common situations.
I don’t think many of you here will be amazed to see that I want to code in Go!
After some pondering, I came up with a different idea. The whole situation is described using several structs embedded in one big struct. All the data, in various forms, are collected in this struct. Hence, these two functions I mentioned can be made methods of this one big struct and return float64 and bool, respectively.
Now it seems so obvious that I am really amazed I did not come up with this idea before. This case study greatly shows how important code design is.
Believe or not, this morning after waking up from a tiring slumber, the following code snippet just came randomly into my mind:
type Data struct {
Float float64
Integer int64
FloatList []float64
IntList []int64
... any other data type that you believe your user will provide ...
}
func MySecretRecipe(input ...*Data) (output *YourOutputType, err error) {
...
}
Not sure will it helps your case since I don’t know about your algorithm. I’m pretty sure user can fill in N number of input and for each input, user can designate any kind of known data type.
As for operating within algorithm, usually I would flatten the Data type into a common one (either using float64 which compromise accuracy for large number capacity OR shifted int64 for accuracy but smaller number capacity). In any cases, big.Int does help as last resort.
Ha! This seems like a very good idea for even a more general scenario than mine. Perfect! I will keep it in mind. First, I will try using one struct because in my situation everything, I think so, can be defined using the data embedded in one struct of several structs. But your idea looks really promising for very general situations. Thanks!
Former Python and Ruby developer here. No worries. Yeah, need change quite a bit. Go is very C-like: detail-oriented and clear.
You won’t regret it.