Using Gson and Retrofit 2 to deserialize complex API responses

Question

I'm using Retrofit 2 and Gson and I'm having trouble deserializing responses from my API. Here's my scenario:

I have a model object named Employee that has three fields: id, name, age.

I have an API that returns a singular Employee object like this:

{
    "status": "success",
    "code": 200,
    "data": {
        "id": "123",
        "id_to_name": {
            "123" : "John Doe"
        },
        "id_to_age": {
            "123" : 30
        }
    }
}

And a list of Employee objects like this:

{
    "status": "success",
    "code": 200,
    "data": [
        {
            "id": "123",
            "id_to_name": {
                "123" : "John Doe"
            },
            "id_to_age": {
                "123" : 30
            }
        },
        {
            "id": "456",
            "id_to_name": {
                "456" : "Jane Smith"
            },
            "id_to_age": {
                "456" : 35
            }
        },
    ]
}

There are three main things to consider here:

1. API responses return in a generic wrapper, with the important part inside of the data field.
2. The API returns objects in a format that doesn't directly correspond to the fields on the model (for example, the value taken from id_to_age needs be mapped to the age field on the model)
3. The data field in the API response can be a singular object, or a list of objects.

How do I implement deserialization with Gson such that it handles these three cases elegantly?

Ideally, I'd prefer to do this entirely with TypeAdapter or TypeAdapterFactory instead of paying the performance penalty of JsonDeserializer. Ultimately, I want to end up with an instance of Employee or List<Employee> such that it satisfies this interface:

public interface EmployeeService {

    @GET("/v1/employees/{employee_id}")
    Observable<Employee> getEmployee(@Path("employee_id") String employeeId);

    @GET("/v1/employees")
    Observable<List<Employee>> getEmployees();

}

This earlier question I posted discusses my first attempt at this, but it fails to consider a few of the gotchas mentioned above: Using Retrofit and RxJava, how do I deserialize JSON when it doesn't map directly to a model object?

Answer 1

I would suggest using a JsonDeserializer because there is not so many levels of nesting in the response, so it won't be a big performance hit.

Classes would look something like this:

Service interface needs to be adjusted for the generic response:

interface EmployeeService {

    @GET("/v1/employees/{employee_id}")
    Observable<DataResponse<Employee>> getEmployee(@Path("employee_id") String employeeId);

    @GET("/v1/employees")
    Observable<DataResponse<List<Employee>>> getEmployees();

}

This is a generic data response:

class DataResponse<T> {

    @SerializedName("data") private T data;

    public T getData() {
        return data;
    }
}

Employee model:

class Employee {

    final String id;
    final String name;
    final int age;

    Employee(String id, String name, int age) {
        this.id = id;
        this.name = name;
        this.age = age;
    }

}

Employee deserializer:

class EmployeeDeserializer implements JsonDeserializer<Employee> {

    @Override
    public Employee deserialize(JsonElement json, Type typeOfT, JsonDeserializationContext context)
            throws JsonParseException {

        JsonObject employeeObject = json.getAsJsonObject();
        String id = employeeObject.get("id").getAsString();
        String name = employeeObject.getAsJsonObject("id_to_name").entrySet().iterator().next().getValue().getAsString();
        int age = employeeObject.getAsJsonObject("id_to_age").entrySet().iterator().next().getValue().getAsInt();

        return new Employee(id, name, age);
    }
}

The problem with the response is that name and age are contained inside of an JSON object whitch translates to a Map in Java so it requires a bit more work to parse it.

Answer 2

EDIT: Relevant update: creating a custom converter factory DOES work--the key to avoiding an infinite loop through ApiResponseConverterFactory's is to call Retrofit's nextResponseBodyConverter which allows you to specify a factory to skip over. The key is this would be a Converter.Factory to register with Retrofit, not a TypeAdapterFactory for Gson. This would actually be preferable since it prevents double-deserialization of the ResponseBody (no need to deserialize the body then repackage it again as another response).

See the gist here for an implementation example.

ORIGINAL ANSWER:

The ApiResponseAdapterFactory approach doesn't work unless you are willing to wrap all your service interfaces with ApiResponse<T>. However, there is another option: OkHttp interceptors.

Here's our strategy:

• For the particular retrofit configuration, you will register an application interceptor that intercepts the Response
• Response#body() will be deserialized as an ApiResponse and we return a new Response where the ResponseBody is just the content we want.

So ApiResponse looks like:

public class ApiResponse {
  String status;
  int code;
  JsonObject data;
}

ApiResponseInterceptor:

public class ApiResponseInterceptor implements Interceptor {
  public static final MediaType JSON = MediaType.parse("application/json; charset=utf-8");
  public static final Gson GSON = new Gson();

  @Override
  public Response intercept(Chain chain) throws IOException {
    Request request = chain.request();
    Response response = chain.proceed(request);
    final ResponseBody body = response.body();
    ApiResponse apiResponse = GSON.fromJson(body.string(), ApiResponse.class);
    body.close();

    // TODO any logic regarding ApiResponse#status or #code you need to do 

    final Response.Builder newResponse = response.newBuilder()
        .body(ResponseBody.create(JSON, apiResponse.data.toString()));
    return newResponse.build();
  }
}

Configure your OkHttp and Retrofit:

OkHttpClient client = new OkHttpClient.Builder()
        .addInterceptor(new ApiResponseInterceptor())
        .build();
Retrofit retrofit = new Retrofit.Builder()
        .client(client)
        .build();

And Employee and EmployeeResponse should follow the adapter factory construct I wrote in the previous question. Now all of the ApiResponse fields should be consumed by the interceptor and every Retrofit call you make should only return the JSON content you are interested in.

Answer 3

Just create following TypeAdapterFactory.

public class ItemTypeAdapterFactory implements TypeAdapterFactory {

  public <T> TypeAdapter<T> create(Gson gson, final TypeToken<T> type) {

    final TypeAdapter<T> delegate = gson.getDelegateAdapter(this, type);
    final TypeAdapter<JsonElement> elementAdapter = gson.getAdapter(JsonElement.class);

    return new TypeAdapter<T>() {

        public void write(JsonWriter out, T value) throws IOException {
            delegate.write(out, value);
        }

        public T read(JsonReader in) throws IOException {

            JsonElement jsonElement = elementAdapter.read(in);
            if (jsonElement.isJsonObject()) {
                JsonObject jsonObject = jsonElement.getAsJsonObject();
                if (jsonObject.has("data")) {
                    jsonElement = jsonObject.get("data");
                }
            }

            return delegate.fromJsonTree(jsonElement);
        }
    }.nullSafe();
}

}

and add it into your GSON builder :

.registerTypeAdapterFactory(new ItemTypeAdapterFactory());

or

 yourGsonBuilder.registerTypeAdapterFactory(new ItemTypeAdapterFactory());

Answer 4

I must say that I haven't thought about using Interceptors for something like this, but that's an interesting approach. Here's what I usually do when I need to model backend-wrapper responses:

If you get something like this from the backend:

{
  "success": "success", // Let's say you may get "error", "unauthorized", etc.
  "payload": [...] // Let's say that you may either get a json object or an array.
}

Then you could declare a deserializer:

import com.demo.core.utils.exceptions.NonSuccessfullResponse
import com.google.gson.Gson
import com.google.gson.JsonDeserializationContext
import com.google.gson.JsonDeserializer
import com.google.gson.JsonElement
import com.google.gson.reflect.TypeToken
import java.lang.reflect.Type

/**
 * A custom deserializers that uses the generic arg TYPE to deserialize on the fly the json responses from
 * the API.
 */
class WrapperDeserializer<TYPE>(
    private val castClazz: Class<TYPE>,
    private val isList: Boolean
) : JsonDeserializer<TYPE> {

    val gson = Gson()

    override fun deserialize(
        element: JsonElement,
        arg1: Type,
        arg2: JsonDeserializationContext
    ): TYPE? {
        val jsonObject = element.asJsonObject

        if (jsonObject.get("success").asBoolean) {
            return if (isList) {
                val type = TypeToken.getParameterized(List::class.java, castClazz).type
                gson.fromJson(jsonObject.get("payload"), type)
            } else {
                gson.fromJson(jsonObject.get("payload"), castClazz)
            }
        } else {
            throw NonSuccessfullResponse()
        }
    }
}

And then in whatever place you instantiate your Gson instance you can do something like:

fun provideGson(): Gson {
        val bookListType = TypeToken.getParameterized(List::class.java, ApiAvailableBooksResponse::class.java).type
        return GsonBuilder()
            .registerTypeAdapter(bookListType, WrapperDeserializer(ApiAvailableBooksResponse::class.java, true))
            .registerTypeAdapter(ApiProfileInfoResponse::class.java, WrapperDeserializer(ApiProfileInfoResponse::class.java, false))
            .registerTypeAdapter(Date::class.java, DateDeserializer())
            .create()
    }

Notice that there we are mapping two different kinds of responses, a list of books, something like:

{
  "success": "success",
  "payload": [
    {...}, // Book 1
    {...}, // Book 2
    {...} // Book 3
  ]
}

And a single user profile response:

{
  "success": "success",
  "payload": {
     "name": "etc",
     // ...
   }
}

Again, the Interceptor approach is a very interesting option that I haven't thought about before - it worries me a bit in terms of flexibility since you're forcing all endpoint responses to follow the same standard - but it looks like a more tidy approach.