Pointer to an interface vs interface holding a pointer when unmarshaling JSON

Question

Consider the following struct and interface definitions.

type Foo interface {
    Operate()
}

type Bar struct {
    A int
}

func (b Bar) Operate() {
    //...
}

Now, if we attempt to perform the following (playground):

var x Foo = Bar{}
err := json.Unmarshal([]byte("{\"a\": 5}"), &x)
fmt.Printf("x: %+v\nerr: %s\n", x, err) 

we get the following output:

x: {A:0}
err: json: cannot unmarshal object into Go value of type main.Foo

However, by replacing the underlying data with the struct type, it goes without a hitch (playground):

var x Foo = &Bar{}
err := json.Unmarshal([]byte("{\"a\": 5}"), &x)
fmt.Printf("x: %+v\nerr: %s\n", x, err)

Output:

x: &{A:5}    
err: %!s(<nil>)

However, this is quite confusing to me. In our call to Unmarshall, we're still passing a pointer to x, which to my understanding, should be sufficient enough to allow us to modify the Bar underneath. After all, pointers are simply addresses in memory. If we're passing that address, we should be able to modify it, no? Why does the second example work, but not the first? Why does the struct being a pointer make all the difference?

Answer 1

The root of the different behavior lies in the fact that if the json package has to create a new value, that value must be a concrete type and cannot be a (non-concrete) interface type.

Let's elaborate this.

First let's examine your 2nd, working example. Your variable x of type Foo wraps a pointer value of type *Bar. When you pass &x to json.Unmarshal(), the json package will get a *Foo pointer value. Pointer to interface! Should not be used, but there it is. The json package will dereference the pointer, and get the wrapped value of type *Bar. Since this is a non-nil pointer, the json package can–and will–go ahead and use it for unmarshaling. All good! The json package will modify the pointed value.

What happens in your first example?

In your first example your x variable of type Foo wraps a non-pointer struct value. Values wrapped in an interface cannot be modified.

What does this mean? The json package will again receive a value of type *Foo. Then it goes ahead and dereferences it, and gets a Bar value wrapped in an interface. The Bar value inside the Foo interface cannot be modified. The only way for the json package to "deliver" the results would be to create a new value that implements Foo, and store this value where the initially passed *Foo points to. But values of Foo cannot be created, it's a non-concrete interface type. So unmarshal returns with an error.

Some addendum. Your 2nd working example:

var x Foo = &Bar{}
err := json.Unmarshal([]byte("{\"a\": 5}"), &x)
fmt.Printf("x: %+v\nerr: %s\n", x, err)

This works, because we "prepare" a non-nil *Bar value to unmarshal into, so the json package doesn't have to create a value itself. We already prepared and passed a value for the Foo interface type (being a value of concrete type *Bar).

If we would store a "typed" nil pointed in x like this:

var x Foo = &Bar{}
x = (*Bar)(nil)
err := json.Unmarshal([]byte("{\"a\": 5}"), &x)
fmt.Printf("x: %+v\nerr: %s\n", x, err)

We will get back the same error (try it on the Go Playground):

x: <nil>
err: json: cannot unmarshal object into Go value of type main.Foo

The explanation is the same: the json package cannot use the nil pointer to unmarshal the value into. It would again need to create a value of non-concrete type Foo, but that is not possible. Only values of concrete types can be created.