Proposal for dynamic implementation of interface

I want to discuss a proposal for adding the following reflection feature to Go.

Given any Go interface type, instantiate a Go value that implements this interface, by passing each method call and its arguments to a user-provided handler.

Some use cases for this feature are:

Debugging and instrumentation

Given any interface A that I use (possibly 3rd party), I want to time each method call and log the calls and the times.

RPC APIs

I could write code that given an interface A, and a struct S that implements A, it would implement HTTP REST client and server components that implement interface A over HTTP, using JSON to marshal method arguments.

The server component would use A and S.
The client component would use only A (the interface), and automatically create a struct that implements it. The server part of this can already be done. But the client cannot be done presently, because there is no way to auto-generate something that implements an interface. I’d have to incorporate a code generator that generates a client stub, which will need to be compiled in the client.

I have done this, but it introduces an extra code generation step in the build process.

This exists in Java

Java does this as follows:

package java.lang.reflect;

interface Invocation Handler {
   Object invoke(Object proxy,Method method,Object[] args)throws Throwable
}

class proxy {
  static newProxyInstance(ClassLoader loader, Class<?>[] interfaces, InvocationHandler h)
}

newProxyInstance() returns an object that implements all the specified interfaces (not just one).

When a method of this object is called, it calls the provided InvocationHandler h.

You can ignore ClassLoader. Go does not load classes dynamically so it does not need a class loader.

Relevant Go features


package reflect

Value.Call(in []Value) []Value

func StructOf(fields []StructField) Type

The Go reflect package has a feature that creates a struct type given any []StructField created at runtime.
Java does not have this.

Proposed Go implementation

I’m not sure exactly what methods such a facility should have, but given the relevant Go features above and the Java implementation, I suggest something like this:


package reflect

ImplementationOf(h func(m Method, in []Value) []Value, u Type) any

Example

Here’s a complete example of how to implement an interface wrapper that prints the execution time of each method call, using the above proposed feature. It includes an implementation of the proposal that is hardcoded to work for this example only.

https://play.golang.com/p/7trKj6TKHsB

package main

import (
	"fmt"
	"reflect"
	"time"
)

type A interface {
	Add(n1, n2 int) int
}

type S struct {
}

func (s *S) Add(n1, n2 int) int {
	return n1 + n2
}

type TimingWrapper struct {
	Type     reflect.Type
	receiver reflect.Value
}

func NewCallWrapper(receiver any) *TimingWrapper {
	var c TimingWrapper
	c.receiver = reflect.ValueOf(receiver)
	c.Type = reflect.TypeOf(receiver)
	return &c
}

func (c *TimingWrapper) Handler(m reflect.Method, in []reflect.Value) []reflect.Value {
	// m is the interface method
	// we need the method for the receiver type
	receiverMethod, ok := c.Type.MethodByName(m.Name)
	if !ok {
		panic(fmt.Sprintf("no such method: %s", m.Name))
	}
	// The interface method does not have a receiver input
	// We need to prepend one:
	rin := make([]reflect.Value, 1, (1 + len(in)))
	rin[0] = c.receiver
	rin = append(rin, in...)

	start := time.Now()
	out := receiverMethod.Func.Call(rin)
	duration := time.Now().Sub(start)
	fmt.Printf("%s: %v\n", m.Name, duration)
	return out
}

func main() {
	var a A = &S{}
	handler := NewCallWrapper(a).Handler
	var p *A
	v := /*reflect.*/ ImplementationOf(handler, reflect.TypeOf(p).Elem())
	a = v.(A)
	d := a.Add(1, 2)
	fmt.Printf("result: %d\n", d)
}

// reflect.ImplementationOf does not exist,
// The code below is an implementation of it that is hardcoded to work for interface A.

type aImpl struct {
	Type    reflect.Type
	Handler func(m reflect.Method, in []reflect.Value) []reflect.Value
}

func (a *aImpl) Add(n1, n2 int) int {
	m, _ := a.Type.MethodByName("Add")
	out := a.Handler(m, []reflect.Value{reflect.ValueOf(n1), reflect.ValueOf(n2)})
	return int(out[0].Int())
}

func ImplementationOf(h func(m reflect.Method, in []reflect.Value) []reflect.Value, u reflect.Type) any {
	var p *A
	a := aImpl{Handler: h, Type: reflect.TypeOf(p).Elem()}
	return &a
}

why do you need reflection for that ? Simply add a wrapper over each method (an object that delegates each method to the actual object, but also times them).

I want to avoid adding these wrapper by hand. Reflection would allow me to do it with less code.
In the example I do it with 4 lines of code for the whole interface. The example interface has only one method, but if an interface has dozens of methods, I would need dozens of wrappers, doing essentially the same thing. The request is to be able to easily create method wrappers that “do essentially the same thing” for all the methods of an interface.

Hi @alex99 ,

You are correct in saying that reflection would make adding the wrappers a bit easier. But even then, you will still be adding a wrapper that measures how long a call takes. Reflection would not eliminate the wrapper objects.

One way to go around this, in case your goal is to debug some code, is to use a profiling library. With a profiling tool you simply run your code, then get to see how much each method took and how many times it was called.

Regarding the reflect proposal, I think it has some merit, maybe other people like it as well. You need to go to the official Golang proposal channels. They have a page for that.

The use case that I care most is automatically building RPC clients. I gave the example of timing method calls because it is simpler.

This would be a cool feature, your should officially propose it. Here are more details on how you can make a Golang proposal GitHub - golang/proposal: Go Project Design Documents

From a system point of view, automatically building RPC clients, you add a bit more complexity to your code. You could achieve the same by having static, predefined clients, and it would make the system a bit less complex.

as useful as this dynamic proxy feature is there are two things that mean it will never be implemented.

  1. it is a “java thing”
  2. “you can just generate the code”

Personally, I used to use it in Java to wrap Type Safe interfaces around Map<String,?> instead of having to map them to yet another type and eat up even more memory and CPU. just to pull a few fields off a JSON blob that I just parsed into a map from a HttpRequest.

I asked about this a few years ago it got shot down without them even understanding why it can be so useful.

I’ve seen your proposal, because I have searched for such a feature: proposal: Go 2: ability to create Dynamic Proxies like in Java · Issue #41897 · golang/go · GitHub
That’s why I want to be careful in how I propose it and try it out here first, before I propose it there.
If I mention Java at all, I would mention it at the end. I first wanted to explain it in Go terms.

I think you didn’t do a good job explaining your use case, and it can be better served with structs instead of interfaces. We could discuss your use case in another topic, but since you mentioned it here, I’ll discuss it below. You can easily convert a JSON map to a struct via the existing json package:

package main

import (
	"encoding/json"
	"fmt"
)

type Power struct {
	Power   float32 `json:"power"`   // current power in W
	Voltage float32 `json:"voltage"` // current voltage in V
	Current float32 `json:"current"` // current current in A
}

func UnmarshalMap(m map[string]any, v any) error {
	// convert the map to JSON first.
	// This is a quick and dirty way to extract a sub-structure from a map.
	// A more efficient way would use reflection directly, but it would be significantly more complicated.
	// You'd need to duplicate a lot of what the json package does already.
	data, err := json.Marshal(m)
	if err != nil {
		return err
	}
	return json.Unmarshal(data, v)
}

func main() {
	m := map[string]any{"power": 10, "voltage": 5, "current": 2}
	var p Power
	err := UnmarshalMap(m, &p)
	if err != nil {
		fmt.Printf("%v\n", err)
		return
	}
	fmt.Printf("%v\n", p)
}

If you want to go directly from the map to any struct, without going through JSON, it is more complicated, but it can be done. I think that would cover your use case: extract a few fields from a map to the public fields of a struct, using reflection (so it works for any struct).

I think you want to be able to do it this way:

package main

import (
	"fmt"
	"reflect"
)

type Power interface {
	Power() float32
	Voltage() float32
	Current() float32
}

func MapToInterface(m map[string]any, t reflect.Type) (any, error) {
	// not possible with Go
	return nil, fmt.Errorf("not implemented")
}

func main() {
	m := map[string]any{"power": 10, "voltage": 5, "current": 2}
	var pp *Power
	v, err := MapToInterface(m, reflect.TypeOf(pp).Elem())
	if err != nil {
		fmt.Printf("%v\n", err)
		return
	}
	p := v.(Power)
	fmt.Printf("power: %f\n", p.Power())
}

This way you can adapt the map to any interface.
I see problems with this:

  • How would you handle formatting errors in one of the fields, if a field is missing from the map, or it cannot be converted to the field type? If the MapToInterface does the conversions, then it is just better to use a struct instead of an interface. If you want to defer the conversion to the time when the methods are called, you may need to add a cumbersome error output to each method. In Java, you can get around this using exceptions that you can catch later. But Go doesn’t use exceptions this way.
  • If you call a method, such as Current() repeatedly, would the conversion happen each time? That is not good. I suppose you could cache each conversion so it only happens once.

For these reasons, I think your use case is better served with structs instead of interfaces.

TL:DR

All that you really need to convince the Go team to implement is a way to get the function you are in as a pointer without having to call runtime.Caller(1) so you can pass to the runtime.FuncForPC() function to introspect it and then do your intercept.

I think you didn’t do a good job explaining your use case,

you would not be wrong :slight_smile:

and it can be better served with structs instead of interfaces.

but it can not, my use case was to make a map[string]interface{} immutable and Type Safe, and that meant an interface with accessors for each of the fields.

I wanted to basically proxy all the functions represented by the interface as look ups in the map.

Then I only needed to maintain code for the one or two fields I needed and could add fields as I needed or the contract they represented changed. I could also convert say string representations of say a ISO8601 TimeStamp to an actual time.Time at the same time.

I was writing 100% immutable Java, and misguided me had not realized that immutability is not as important in Go because everything is a Value type and is copied by default.

And any errors would just be a panic, I am also a former Erlang main, as well a former Java main. 0;-)

I did figure out how to implement Dynamic Proxy behavior in Go, but it is not really viable for a couple reasons.

You can introspect a function and call another function in its place just like Dynamic Proxies let you do, but you have to pass the function you are intercepting as an argument.

There is only one way to get the “current” function name and that is to basically pull a stack trace, or at least make a call that walks the stack and is just as expensive. It makes normal reflect package calls look performant.

runtime.Caller(1)

Caller reports file and line number information about function invocations on the calling goroutine’s stack. The argument skip is the number of stack frames to ascend, with 0 identifying the caller of Caller.

pc, _, _, _ := runtime.Caller(1)
return fmt.Sprintf(%s, runtime.FuncForPC(pc).Name())

and for receiver methods like you would have on a bare struct with an interface with just accessor functions, but in this version you can skip the runtime.Caller() call if you know the name. Which I would if I was generating this code, but then, if I an generating this code, why not just generate the lookup and type assertion instead of all this round about mess.

runtime.FuncForPC()

FuncForPC returns a *Func describing the function that contains the given program counter address, or else nil.

func (m myStruct) MyFunc() {
	// Get the name of the current function.
	fname := runtime.FuncForPC(reflect.ValueOf(m.MyFunc).Pointer()).Name()

	// Look up the function in the map.
	fn, ok := fns[fname]
	if !ok {
		panic(fmt.Sprintf("could not find %s", fname))
	}

	// Call the function.
	fn()
}

I got it to “work”, but in the end, it was way more boilerplate and trouble and cost in time and space than it was worth.

This could then be represented as a Monadic function wrapper that, in my case looked at the function name, converted it to a key to look up in the the backing map and then return the value and assert it to the type that the original function was expecting to be returned if needed.

You should really start another topic if you want to discuss your use case further.
I don’t understand what you’re trying to do. Perhaps a code example would make it clearer.

Go can call methods dynamically via reflection, but it cannot implement interfaces dynamically. The feature I propose is to be able to implement an arbitrary interface dynamically.

You can have immutable data structures in Go just like you can do in Java. Interfaces are one way to do it. Passing small structures by value is another way to do it in Go, that Java does not have. If you pass something by value to a function, the called function cannot modify your copy of the value.

Examples of immutability in Go:

  • reflect.Type is an interface that points to something immutable.
  • reflect.Type.MethodByName(string) (Method, bool). It returns Method by value, so if you modify it you are not modifying anything in the Type.

implementing an interface dynamically at runtime is exactly what this is, not sure why no one seems to understand it … the implementation is just to delegate to a map, not such an obtuse usecase.

anyway, after I spent more time with Go and I got it working; I realized it was a pretty useless feature to have in Go because Go is Not Java and there are better ways to accomplish the dynamic interface implementation at runtime.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.