Deserializing JSON using Java 11 HttpClient and custom BodyHandler with Jackson halts and will not proceed

Question

I have a problem with deserializing JSON to custom object directly using Java 11 HttpClient::send with custom HttpResponse.BodyHandler. I came across this issue while answering this SO question.

Versions that I am using :

OpenJDK 11
Jackson 2.9.9.3

I created a simple generic JsonBodyHandler class which implements HttpResponse.BodyHandler:

public class JsonBodyHandler<W> implements HttpResponse.BodyHandler<W> {

    private final Class<W> wClass;

    public JsonBodyHandler(Class<W> wClass) {
        this.wClass = wClass;
    }

    @Override
    public HttpResponse.BodySubscriber<W> apply(HttpResponse.ResponseInfo responseInfo) {
        return asJSON(wClass);
    }

}

the asJSON method is defined as :

public static <W> HttpResponse.BodySubscriber<W> asJSON(Class<W> targetType) {
        HttpResponse.BodySubscriber<String> upstream = HttpResponse.BodySubscribers.ofString(StandardCharsets.UTF_8);

        return HttpResponse.BodySubscribers.mapping(
                upstream,
                (String body) -> {
                    try {
                        ObjectMapper objectMapper = new ObjectMapper();
                        return objectMapper.readValue(body, targetType);
                    } catch (IOException e) {
                        throw new UncheckedIOException(e);
                    }
            });
}

So it returns a custom HttpResponse.BodySubscriber which gets body as String and then applies mapping from JSON to given targetType The code to test it :

public static void main(String[] args) throws URISyntaxException, IOException, InterruptedException {

        HttpRequest request = HttpRequest.newBuilder(new URI("https://jsonplaceholder.typicode.com/todos/1"))
                .header("Accept", "application/json")
                .build();

        Model model = HttpClient.newHttpClient()
                .send(request, new JsonBodyHandler<>(Model.class))
                .body();

        System.out.println(model);

}

And the Model class :

public class Model {
        private String userId;
        private String id;
        private String title;
        private boolean completed;


    //getters setters constructors toString
}

The output is as expected :

Model{userId='1', id='1', title='delectus aut autem', completed=false}

However when I change the asJSON method to read InputStream instead of String first :

public static <W> HttpResponse.BodySubscriber<W> asJSON(Class<W> targetType) {
    HttpResponse.BodySubscriber<InputStream> upstream = HttpResponse.BodySubscribers.ofInputStream();

    return HttpResponse.BodySubscribers.mapping(
                upstream,
                (InputStream is) -> {
                    try (InputStream stream = is) {
                        ObjectMapper objectMapper = new ObjectMapper();
                        return objectMapper.readValue(stream, targetType);
                    } catch (IOException e) {
                        throw new UncheckedIOException(e);
                    }
            });
}

It hangs after invoking reading the value with ObjectMapper and it does not proceed (I have checked that it successfully gets the response from the endpoint, status code is 200) but then it just hangs. Does anyone know what might be the issue?

Have you disabled the auto close source feature? JsonParser.Feature.AUTO_CLOSE_SOURCE. Suspecting that the stream might be open — Arpan Kanthal, Aug 23 '19 at 16:02
I have not done it but `objectMapper.disable(JsonParser.Feature.AUTO_CLOSE_SOURCE)` did not help. This feature is enabled by default as I see - when I remove `try-with-resources` it does not work neither (Jackson should close the stream because of this feature) — Michał Krzywański, Aug 23 '19 at 16:08

score 22 · Accepted Answer · edited Nov 19 '19 at 17:31

I have just found this SO question which has the same problem but with GZIPInputStream. It turns out that HttpResponse.BodySubscribers.mapping is buggy and it does not work as documented. Here is the link to the official OpenJDK bug site. It was fixed for OpenJDK 13. So one workaround is to use HttpResponse.BodySubscribers::ofString instead of HttpResponse.BodySubscribers::ofInputStream as upstream for HttpResponse.BodySubscribers::mapping - it is shown how to do it in my question.

Or a better solution to this, as pointed by @daniel in comment, is to return a Supplier instead of model class:

public static <W> HttpResponse.BodySubscriber<Supplier<W>> asJSON(Class<W> targetType) {
        HttpResponse.BodySubscriber<InputStream> upstream = HttpResponse.BodySubscribers.ofInputStream();

        return HttpResponse.BodySubscribers.mapping(
                upstream,
                inputStream -> toSupplierOfType(inputStream, targetType));
    }

    public static <W> Supplier<W> toSupplierOfType(InputStream inputStream, Class<W> targetType) {
        return () -> {
            try (InputStream stream = inputStream) {
                ObjectMapper objectMapper = new ObjectMapper();
                return objectMapper.readValue(stream, targetType);
            } catch (IOException e) {
                throw new UncheckedIOException(e);
            }
        };
    }

The JsonBodyHandler also uses Supplier:

public class JsonBodyHandler<W> implements HttpResponse.BodyHandler<Supplier<W>> {

    private final Class<W> wClass;

    public JsonBodyHandler(Class<W> wClass) {
        this.wClass = wClass;
    }

    @Override
    public HttpResponse.BodySubscriber<Supplier<W>> apply(HttpResponse.ResponseInfo responseInfo) {
        return asJSON(wClass);
    }

}

And then we can call it like this:

public static void main(String[] args) throws URISyntaxException, IOException, InterruptedException {

    HttpRequest request = HttpRequest.newBuilder(new URI("https://jsonplaceholder.typicode.com/todos/1"))
            .header("Accept", "application/json")
            .build();

    Model model = HttpClient.newHttpClient()
            .send(request, new JsonBodyHandler<>(Model.class))
            .body()
            .get();

    System.out.println(model);

}

This is even promoted way to do it described in OpenJDK 13 docs):

The mapping function is executed using the client's executor, and can therefore be used to map any response body type, including blocking InputStream. However, performing any blocking operation in the mapper function runs the risk of blocking the executor's thread for an unknown amount of time (at least until the blocking operation finishes), which may end up starving the executor of available threads. Therefore, in the case where mapping to the desired type might block (e.g. by reading on the InputStream), then mapping to a Supplier of the desired type and deferring the blocking operation until Supplier::get is invoked by the caller's thread should be preferred.

Or you could map to a `Supplier` instead of mapping to `Model` which would delay the blocking call to `InputStream::read` until it's safe to do so. Though the bug has been fixed in JDK 13 - calling blocking operations from the mapping function is still discouraged, and the documentation has been updated accordingly: https://download.java.net/java/early_access/jdk13/docs/api/java.net.http/java/net/http/HttpResponse.BodySubscribers.html#mapping(java.net.http.HttpResponse.BodySubscriber,java.util.function.Function) — daniel, Aug 26 '19 at 11:01

score 9 · Answer 2 · answered Sep 16 '21 at 15:23

Using Jackson TypeReference I was able to do it with generics alone, not requiring a redundant Class<T> parameter.

package com.company;

import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.ObjectMapper;

import java.io.IOException;
import java.io.InputStream;
import java.io.UncheckedIOException;
import java.net.http.HttpResponse;
import java.util.function.Supplier;

public class JsonBodyHandler<W> implements HttpResponse.BodyHandler<Supplier<W>> {

    public JsonBodyHandler() {
    }

    private static <W> Supplier<W> toSupplierOfType(InputStream inputStream) {
        return () -> {
            try (InputStream stream = inputStream) {
                return new ObjectMapper().readValue(stream, new TypeReference<W>() {
                });
            } catch (IOException e) {
                throw new UncheckedIOException(e);
            }
        };
    }

    private static <W> HttpResponse.BodySubscriber<Supplier<W>> asJSON() {
        return HttpResponse.BodySubscribers.mapping(
                HttpResponse.BodySubscribers.ofInputStream(),
                JsonBodyHandler::toSupplierOfType);
    }

    @Override
    public HttpResponse.BodySubscriber<Supplier<W>> apply(HttpResponse.ResponseInfo responseInfo) {
        return JsonBodyHandler.asJSON();
    }

}

And in use

package com.company;

import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.time.Duration;
import java.util.List;


public class Main {

    private static final HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create("https://myApi"))
            .timeout(Duration.ofMinutes(1))
            .header("Content-Type", "application/json")
            .build();

    private static final HttpClient client = HttpClient.newBuilder().build();

    public static void main(String[] args) throws InterruptedException {
        client.sendAsync(Main.request, new JsonBodyHandler<List<MyDto>>())
                .thenAccept(response -> {
                    List<MyDto> myDtos = response.body().get();
                    System.out.println(myDtos);
                }).join();
    }
}

One quick note: code above is creating new ObjectMapper for each read request. That's hugely inefficient, so ideally a static instance is created and used instead. — StaxMan, Mar 17 '23 at 16:56

score 2 · Answer 3 · answered May 28 '21 at 12:45

in kotlin:

    val objectMapper = ObjectMapper()

    fun jsonBodyHandler(): HttpResponse.BodyHandler<JsonNode> {
      val jsonNodeSubscriber = BodySubscribers.mapping(BodySubscribers.ofByteArray()) {
        objectMapper.readTree(it)
      }
      return HttpResponse.BodyHandler { jsonNodeSubscriber }
    }

Deserializing JSON using Java 11 HttpClient and custom BodyHandler with Jackson halts and will not proceed

3 Answers3

Linked