Weirdness related to interface{} and pointers

[yihangho]

I am writing a library that consumes some API that responds with JSON. Obviously, my library should parse this JSON response into something friendlier, like a struct. However, I want to give my user the flexibility to define their own structs (for e.g., to cater for different naming scheme). To do so, the user should provide a template struct, which will be used to parse the JSON response. Since the type of this user-provided struct is not known, we have to use interface{}.

Here is my attempt:

package library

import (
    "encoding/json"
)

// Assume that the API always returns with this
var mockResponse = []byte(`{ "id": 123 }`)
var Template interface{}

func FetchData() interface{} {
    // Do something to fetch the data
    // Assume that JSON deserialization always works
    _ = json.Unmarshal(mockResponse, &Template)
    return Template
}

Ideally, this is how I want my user to use the library:

package main

import (
    "library"
)

type MyStruct struct {
    ID int `json:"id"`
}

func main() {
    library.Template = MyStruct{}
    val := library.FetchData()
    // And then use type assertion to cast `val` into MyStruct
}

Interestingly, if we do this, val becomes a map[string][]interface{}, which is not what we want. By some trial and error, if I were to do library.Template = &MyStruct{} instead, val would be a *MyStruct. Close enough.

Here’s a replication of this: https://repl.it/@yihangho/Weird-interface-and-pointers

How do I reason about this weirdness?

[clbanning (Charles Banning)]

You probably want to structure your code something simpler like https://play.golang.org/p/qec5R1fkkJO.

[yihangho]

That’s a reasonable suggestion. Thanks! But still curious why the code I posted behaves like it does.

[Yamil_Bracho]

In the Go packages dpcumentacion, in json. UnMarshall we found
To unmarshal JSON into an interface value, Unmarshal stores one of these in the interface value:

bool, for JSON booleans
float64, for JSON numbers
string, for JSON strings
[]interface{}, for JSON arrays
map[string]interface{}, for JSON objects
nil for JSON null

[clbanning (Charles Banning)]

Also, per https://golang.org/pkg/encoding/json/#Unmarshal:

To unmarshal JSON into a pointer, Unmarshal first handles 
the case of the JSON being the JSON literal null. In that case, 
Unmarshal sets the pointer to nil. Otherwise, Unmarshal 
unmarshals the JSON into the value pointed at by the pointer. 
If the pointer is nil, Unmarshal allocates a new value for it
to point to.

If you review the code in go/src/encoding/json/decode.go (lines 428-432) you’ll see that the first non-pointer encountered while walking down ‘v’ is used as the type of the value.

“Template = MyStruct{}” is of type interface and &Template is a pointer to an interface{}, which doesn’t itself resolve to a pointer; so the JSON object is unmarshaled to a map[string]interface{} value.

"Template = &MyStruct({} is of type interface{} that does resolve to a pointer, so it “walks down” through it to MyStruct{} and that is the type of ‘v’, so the JSON object is unmarshaled to MyStruct{}.

[skillian (Sean Killian)]

Hi, yihangho,

I just wanted to provide another example of what people in this thread are already saying. One thing that helps me understand why what you’re trying to do isn’t working is to write up a trivial Unmarshal function of my own and try to handle different values passed in:

https://play.golang.org/p/QXUyYn-eoWT

@clbanning has already explained the difference between the types of values being passed in, but I thought it might be helpful to see from the “other side” of unmarshaling.