Parse a JSONNode using Java 8 stream API

Question

I'm reading all Objects from Salesforce environment using Java, it's working fine but below code is taking 10 mins to convert the JSON into Java ArrayList. I was thinking if I can use Java - 8 stream API to parallel the parsing logic. Below is my working code, any suggestion appreciated.

/**
 * @Desc : Get All available objects(tables) from salesforce
 * @return : List<SalesforceObject>
 * */
public List<SalesforceObject> getAllsObjects() {

    List<SalesforceObject> listsObject = new ArrayList<SalesforceObject>();

    try {

        // query Salesforce
        final URIBuilder builder = new URIBuilder(this.sfAccess.instanceURL);
        builder.setPath(appProp.salesforceObjectPath);

        final HttpGet get = new HttpGet(builder.build());
        get.setHeader("Authorization", "Bearer " + this.sfAccess.token);

        final CloseableHttpClient httpclient = HttpClients.createDefault();
        final HttpResponse queryResponse = httpclient.execute(get);

        // parse
        final ObjectMapper mapper = new ObjectMapper().enable(SerializationFeature.INDENT_OUTPUT);
        final JsonNode queryResults = mapper.readValue(queryResponse.getEntity().getContent(), JsonNode.class);
        System.out.println(queryResults);

        // This line takes - 10 mins
        listsObject.addAll(mapper.convertValue(queryResults.get("sobjects"), new TypeReference<List<SalesforceObject>>(){}));

        return listsObject;
    } catch(IOException e) {
        e.printStackTrace();
    } catch (URISyntaxException e) {
        e.printStackTrace();
    }

    return null;
}

which library is this `JsonNode` from? – Ryuzaki L Sep 16 '19 at 17:46 — Ryuzaki L, Sep 16 '19 at 17:46
com.fasterxml.jackson.databind.JsonNode; – Roul Sep 17 '19 at 09:09 — Roul, Sep 17 '19 at 09:09

score 4 · Accepted Answer · answered Sep 18 '19 at 18:55

4

You are looking at,

return StreamSupport.stream(queryResults.get("sobjects").spliterator(), true)
            .map(sObj -> mapper.convertValue(sObj, SalesforceObject.class))
            .collect(Collectors.toList());

Note that your concurrency will still be limited by the number of CPU cores of your server.

answered Sep 18 '19 at 18:55

Prasanna

2,593
7
39
53

queryResults.get("sobjects") this line is taking more time it seems.. the data size 300KB around. – Roul Sep 19 '19 at 06:54
It was taking more time because i was printing the result in console and size it 4 MB.. Thanks for your help! – Roul Sep 19 '19 at 07:16

Parse a JSONNode using Java 8 stream API

1 Answers1