What is the best way to load different interface implementations at runtime in Go?

Getting Help
1

Hi everyone :grinning_face:
Quick question about Go, and I’d love your practical opinions/war stories.

Background / goal
I have a core repo (call it repo A) that defines a FileService interface that will be used throughout my application:

type FileService interface {
    Upload(path string, data []byte) (url string, err error)
    Delete(url string) error
}

Concrete implementations should live in a separate repo (call it repo B): e.g. azure_fileservice and s3_fileservice. A deployer should be able to pick which implementation to use via configuration at runtime. Ideally the app in repo A would load the chosen implementation on startup.

My current idea
Use Go’s plugin mechanism: build each implementation as a .so plugin and load the chosen plugin at runtime using plugin.Open() and a well-known exported symbol (e.g. New or NewFileService). That way repo A stays unchanged and deployers can drop-in a plugin file for the impl they want.

I need to know whether you think this is feasible and is the best way to handle such a setup, or if there are better approaches out there. Please share your thoughts and experiences. I’d really appreciate hearing what’s worked (or not worked) for you.

Thank you in advance! :grinning_face:

2

You could, but plugin package - plugin - Go Packages. I’ve never used plugins because of these, mostly the first one. If you can live with some tighter coupling, then make your implementations modules and import all of them in the main, and instantiate the desired implementation based on the configuration. You trade-off a little flexibility and danger, for less flexibility and safety.

1 Like
3

What’s your motivation here? Are you trying to make smaller binary sizes? From the docs:

  • Together, these restrictions mean that, in practice, the application and its plugins must all be built together by a single person or component of a system. In that case, it may be simpler for that person or component to generate Go source files that blank-import the desired set of plugins and then compile a static executable in the usual way.

For these reasons, many users decide that traditional interprocess communication (IPC) mechanisms such as sockets, pipes, remote procedure call (RPC), shared memory mappings, or file system operations may be more suitable despite the performance overheads.

It might be better to just import the libraries you need. There’s some discussion here:

Editing to add: gRPC might be a good option for you:

1 Like
4

If you only have a few alternatives, you could use build flags, which does mean compiling (or running) a version for a specific implementation - this saves on run time resources, e.g.

  1. add package (directory) fileservice
  2. add s3_fileservice.go with
//go:build s3
...
package fileservice
...
  1. similarly for azure_fileservice.go
  2. – likely you will also have a fileservice.go that is for both
  3. Use import "fileservice" to import
  4. use command line, e.g. go build -o s3.exe -tags=s3 .
  5. – or for VS code, you can add build tags to your compile/run, e.g.
"configurations": [
    {
        "name": "S3",
            "type": "go",
            "buildFlags": "-tags='s3'",
...

N.B. this is compile time, not runtime, but does allow you to have lighter weight (and safer?) executables which only include the relevant code.

There’s more on build tags here - go command - cmd/go - Go Packages but it’s not great for your own tags - also try here - Decoding Go Language Source Code: A Deep Dive into go: Directives and the Use of Automation Tools | Xinwei Xiong(cubxxw)'s English blog

Build tags can be combined using || and && and also !

Hope this helps - Andy