Can not read large JSON data from inputstream

Question

Java(TM) SE Runtime Environment (build 1.7.0_45-b18)

Hello,

I am using the httpUrlConnection to retrieve a json string from a webservice. Then I get the inputStream from the connection

jsonString = readJSONInputStream(mHttpUrlconnection.getInputStream());

I then use the following function to read the inputstream to get the JSON.

private String readJSONInputStream(final InputStream inputStream) {
    log.log(Level.INFO, "readJSONInputStream()");

    Reader reader = null;

    try {
        final int SIZE = 16092;

        char[] buffer = new char[SIZE];
        int bytesRead = 0;
        int read = 0;

        reader = new BufferedReader(new InputStreamReader(inputStream, "UTF-8"), SIZE);
        bytesRead = reader.read(buffer);

        String jsonString = new String(buffer, 0, bytesRead);
        log.log(Level.INFO, "bytesRead: " + bytesRead + " [ " + jsonString + " ]");
        /* Success */
        return jsonString;
    }
    catch(IndexOutOfBoundsException ex) {
        log.log(Level.SEVERE, "UnsupportedEncodingexception: " + ex.getMessage());
    }
    catch(IOException ex) {
        log.log(Level.SEVERE, "IOException: " + ex.getMessage());
    }
    finally {
        /* close resources */
        try {
            reader.close();
            inputStream.close();
        }
        catch(IOException ex) {
            log.log(Level.SEVERE, "IOException: " + ex.getMessage());
        }
    }

    return null;
}

However, if the json is small say 600 bytes then everything is ok (so the code above does work for smaller data). But I have some JSON that is about 15000 bytes in size so I set the maximum size to 16092.

However, the JSON it only reads about about 6511 and just cuts off.

I don't understand that if the JSON is small there is no problem. But for the larger JSON it just cuts off at the same size each time.

Am I doing anything wrong here. Anything I should check.

Many thanks for any suggestions,

Aren't there size restrictions on HTTP GET requests? https://stackoverflow.com/questions/2659952/maximum-length-of-http-get-request — duffymo, Nov 18 '15 at 11:30
I am using POST not GET i.e. mHttpUrlConnection.setRequestMethod("POST"); — ant2009, Nov 18 '15 at 11:33

score 3 · Accepted Answer · edited Jul 25 '18 at 00:54

3

Try the following code:

String readResponse(InputStream inputStream) throws IOException {
    BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
    String line = "";
    String result = "";
    while ((line = bufferedReader.readLine()) != null)
        result += line;

    inputStream.close();
    return result;
}

edited Jul 25 '18 at 00:54

Pang

9,564
146
81
122

answered Nov 18 '15 at 11:31

Arpit Agrawal

321
1
2
14

Testing your code now. Should you not test for -1 instead of null in the while loop? – ant2009 Nov 18 '15 at 11:57
It is working totally fine, I have used the same code many times. Is it causing you a trouble?? – Arpit Agrawal Nov 18 '15 at 12:01
The return is still truncated/cut for me – Honest Objections Sep 03 '17 at 23:30

score 2 · Answer 2 · answered Nov 18 '15 at 11:32

2

Most likely your code is broken here:

bytesRead = reader.read(buffer);

You should have cycle around read. I think you do not read all chars from stream.

answered Nov 18 '15 at 11:32

sibnick

3,995
20
20

score 1 · Answer 3 · answered Nov 18 '15 at 11:37

You should continue to read from the buffer until bytesRead = -1.

I suspect that the reader is buffering the data and only returning a certain amount of data at a time.

Try looping until bytesRead == -1 with each loop append the read data to your resultant String.

Can not read large JSON data from inputstream

3 Answers3