Golang Marshal/Unmarshal json with a custom tag

Question

I want to Marshal / Unmarshal Golang object (json) with a custom tag.

Like

type Foo struct {
    Bar string `json:"test" es:"bar"`
}

data, _ := json.MarshalWithESTag(Foo{"Bar"})
log.Println(string(data)) // -> {"foo":"bar"}

In other words, I whan to use the encoding/json library with a different tag here: https://github.com/golang/go/blob/master/src/encoding/json/encode.go#L1033

Thanks :)

Answer 1

I think the way you wrote your example might have been a bit incorrect?

When I run your code using Marshal() inplace of MarshalWithESTag() I get {"test":"Bar"} not {"foo":"test"} as I think your example would imply. Here is that code running in the Go Playground to illustrate the output:

package main

import (
    "encoding/json"
    "fmt"
)
type Foo struct {
    Bar string `json:"test" es:"bar"`
}
func main() {
    data, _ := json.Marshal(Foo{"Bar"})
    fmt.Println(string(data))
}

Assuming I am correct about what you wanted then that would imply what you really wanted was for your output to be {"bar":"Bar"} when you call json.MarshalWithESTag().

Based on that assumption you could accomplish with the following code — which you can see in the Go Playground — after which I will explain the code. (If my assumption was not correct I will address that too):

1. You cannot add a MarshalWithESTag() method to the the json package because Go does not allow for safe monkey patching. However, you can add a MarshalWithESTag() method to your Foo struct, and this example also shows you how to call it:

   func (f Foo) MarshalWithESTag() ([]byte, error) {
       data, err := json.Marshal(f)
       return data,err
   }
   
   func main()  {
       f := &Foo{"Bar"}
       data, _ := f.MarshalWithESTag()
       log.Println(string(data)) // -> {"bar":"Bar"}
   }
   

2. Next you need to add a MarshalJSON() method to your Foo struct. This will get called when you call json.Marshal() and pass an instance of Foo to it.
   
   The following is a simple example that hard-codes a return value of {"hello":"goodbye"} so you can see in the playground how adding a MarshalJSON() to Foo affects json.Marshal(Foo{"Bar"}):

   func (f Foo) MarshalJSON() ([]byte, error) {
       return []byte(`{"hello":"goodbye"}`),nil
   }
   

   The output for this will be:

   {"hello":"goodbye"}
   

3. Inside the MarshalJSON() method we need to produce JSON with the es tags instead of the json tags meaning we will need to generate JSON within the method because Go does not provide us with the JSON; it expects us to generate it.
   
   And the easiest way to generate JSON in Go is to use json.Marshal(). However, if we use json.Marshal(f) where f is an instance of Foo that gets passed as the receiver when calling MarshalJson() it will end up in an infinite recursive loop!
   
   The solution is to create a new struct type based on and identical to the existing type of Foo, except for its identity. Creating a new type esFoo based on Foo is as easy as:

   type esFoo Foo
   

4. Since we have esFoo we can now cast our instance of Foo to be of type esFoo to break the association with our custom MarshalJSON(). This works because our method was specific to the type with the identity of Foo and not with the type esFoo. Passing an instance of esFoo to json.Marshal() allows us to use the default JSON marshalling we get from Go.
   
   To illustrate, here you can see an example that uses esFoo and sets its Bar property to "baz" giving us output of {"test":"baz"} (you can also see it run in the Go playground):

   type esFoo Foo
   func (f Foo) MarshalJSON() ([]byte, error) {
       es := esFoo(f)
       es.Bar = "baz"
       _json,err := json.Marshal(es)
       return _json,err
   }
   

   The output for this will be:

   {"test":"baz"}
   

5. Next we process and manipulate the JSON inside MarshalJSON(). This can be done by using json.Unmarshal() to an interface{} variable which we can then use a type assertion to treat the variable as a map.
   
   Here is a standalone example unrelated to the prior examples that illustrates this by printing map[maker:Chevrolet model:Corvette year:2021] (Again you can see it work in the Go Playground):

   package main
   
   import (
       "encoding/json"
       "fmt"
   )
   type Car struct {
       Maker string `json:"maker" es:"fabricante"`
       Model string `json:"model" es:"modelo"`
       Year  int    `json:"year"  es:"año"`    
   }
   var car = Car{
       Maker:"Chevrolet",
       Model:"Corvette",
       Year:2021,
   }
   
   func main() {
       _json,_ := json.Marshal(car)
       var intf interface{}
       _ = json.Unmarshal(_json, &intf)
       m := intf.(map[string]interface{})      
       fmt.Printf("%v",m)
   }
   

   The output for this will be:

   map[maker:Chevrolet model:Corvette year:2021]
   

6. Our next challenge is to access the tags. Tags are accessible using Reflection. Go provides reflection functionality in the standard reflect package.
   
   Using our Car struct from above, here is a simple example that illustrates how to use Reflection. It uses the reflect.TypeOf() function to retrieve the type as a value and then introspects that type to retrieve the tags for each field. The code for retrieving each tag is t.Field(i).Tag.Lookup("es"), which is hopefully somewhat self-explanatory (and again, check it out in the Go Playground):

   func main() {
       t := reflect.TypeOf(car)    
       for i:=0; i<t.NumField();i++{
           tag, _ := t.Field(i).Tag.Lookup("es")
           fmt.Printf("%s\n",tag)
       }
   }
   

   The output for this will be:

   fabricante
   modelo
   año
   

7. Now that we have covered all the building blocks we can bring it all together into a working solution. The only addition worth mentioning are the creation of a new map variable _m of the same length as m to allow us to store the values using the es tags:

   func (f Foo) MarshalJSON() ([]byte, error) {
       es := esFoo(f)
       _json,err := json.Marshal(es)
       {
           if err != nil {
               goto end
           }
           var intf interface{}
           err = json.Unmarshal(_json, &intf)
           if err != nil {
               goto end
           }
           m := intf.(map[string]interface{})
           _m := make(map[string]interface{},len(m))
           t := reflect.TypeOf(f)
           i := 0
           for _,v := range m {
               tag, found := t.Field(i).Tag.Lookup("es")
               if !found {
                   continue
               }
               _m[tag] = v
               i++
           }
           _json,err = json.Marshal(_m)
       }
   end:
       return _json,err
   }
   

8. However, there is still one detail left undone. With all the above code f.MarshalWithESTag() will generate JSON for the es tags, but so will json.Marshal(f) and we want the latter to return its use of the json tags.
   
   So address that we just need to:

   a. Add a local package variable useESTags with an initial value of false,

   b. Modify f.MarshalWithESTag() to set useESTags to true before calling json.Marshal(), and then

   c. To set useESTags back to false before returning, and

   d. Lastly modify MarshalJSON() to only perform the logic required for the es tags if useESTags is set to true:
   
   Which brings us to the final code — with a second property in Foo to provide a better example (and finally, you can of course see here in the Go Playground):

   package main
   
   import (
       "encoding/json"
       "log"
       "reflect"
   )
   
   type Foo struct {
       Foo string `json:"test" es:"bar"`
       Bar string `json:"live" es:"baz"`
   }
   type esFoo Foo
   var useESTags = false
   func (f Foo) MarshalWithESTag() ([]byte, error) {
       useESTags = true
       data, err := json.Marshal(f)
       useESTags = false
       return data,err
   }
   func (f Foo) MarshalJSON() ([]byte, error) {
       es := esFoo(f)
       _json,err := json.Marshal(es)
       if useESTags {
           if err != nil {
               goto end
           }
           var intf interface{}
           err = json.Unmarshal(_json, &intf)
           if err != nil {
               goto end
           }
           m := intf.(map[string]interface{})
           _m := make(map[string]interface{},len(m))
           t := reflect.TypeOf(f)
           i := 0
           for _,v := range m {
               tag, found := t.Field(i).Tag.Lookup("es")
               if !found {
                   continue
               }
               _m[tag] = v
               i++
           }
           _json,err = json.Marshal(_m)
       }
   end:
       return _json,err
   }
   
   func main()  {
       f := &Foo{"Hello","World"}
       data, _ := json.Marshal(f)
       log.Println(string(data)) // -> {"test":"Hello","live":"World"}
       data, _ = f.MarshalWithESTag()
       log.Println(string(data)) // -> {"bar":"Hello","baz":"World"}
   }
   

Epilogue

1. If my assumption was wrong I think I can at least assume this code I provided gives you enough to achieve your objective. You should be able to swap the keys and values in your output if that is actually what you want given the techniques shown. If not, please comment asking for help.

2. Finally, I would be remiss not to mention that reflection can be slow and that this example uses reflection multiple times per object to achieve your desired output. For many use-cases the time required to process JSON this way won't be significant. However, for many other use-cases the execution time can be a deal-killer. Several commented that you should approach this a different way; if performance matters and/or using a more idiomatic Go approach is important, you might want to seriously consider their recommendations.