JSON Serialisation from URL always returning NULL

Question

I have a web URL which returns a JSON formatted string on request

{"StockID":0,"LastTradePriceOnly":"494.92","ChangePercent":"0.48"}

I'm streaming this using Java

InputStream in = null;
in = url.openStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
StringBuilder sb = new StringBuilder();

String line = null;
try {
     while ((line = reader.readLine()) != null) {
     sb.append(line + "\n");
     }


} 
catch (IOException e) {
                e.printStackTrace();
            } finally {
                try {
                    in.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
            String result = sb.toString();

But reader.readLine() is always returning null

Any idea what I'm doing wrong here?

Here is the actual JSON address http://app.myallies.com/api/quote/goog

UPDATE

Same code works OK on http://app.myallies.com/api/news, although both links have the same server implementation to generate the JSON response.

Google has some nice serialization libraries for POJO's. https://code.google.com/p/google-gson/ — Timothy Frisch, Jan 13 '15 at 19:48
If it's just an empty reader, have you tried flushing it? Also, this: http://stackoverflow.com/questions/4308554/simplest-way-to-read-json-from-a-url-in-java — Aify, Jan 13 '15 at 19:50
Updated question, same code works well on another JSON resource on the server — Maya, Jan 13 '15 at 19:54
Not sure if this is why, but the StringBuilder default constructor has a capacity of 16 chars. I've never used SB before, so I don't know if appending something longer than the limit will break it, but that's just a thought. Should take a look into it. — Aify, Jan 13 '15 at 20:25

score 2 · Accepted Answer · answered Jan 13 '15 at 21:06

It looks like it was the User-Agent that it wanted. The following code works for me:

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;

public class JSONTest {

    public static void main(String[] args) throws Exception {

        URL url = new URL("http://app.myallies.com/api/quote/goog");

        HttpURLConnection connection = (HttpURLConnection) url.openConnection();
        connection.setRequestProperty("User-Agent", "Mozilla/5.0 (X11; Linux x86_64; rv:33.0) Gecko/20100101 Firefox/33.0");
        connection.setDoInput(true);

        BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
        StringBuilder sb = new StringBuilder();

        String line = null;
         while ((line = reader.readLine()) != null) {
             sb.append(line + "\n");
         }

         System.out.println(sb.toString());

    }

}

Wow you actually right! I have checked the documentation and it looks like you have to send a special flag to do with the user agent to indicate you are not a bot or a search engine. Thank you so much I wish I could upvote twice! — Maya, Jan 13 '15 at 21:20

score 0 · Answer 2 · answered Jan 13 '15 at 20:35

The reason you are getting null is that the request that Java makes is not identical to the request that your browser makes. Your browser includes a lot of request headers that Java won't populate unless you do so explicitly. I'm not sure why the webserver doesn't respond to the basic request, though. You can test this yourself by doing the following:

Basic request (no headers):

curl http://app.myallies.com/api/quote/goog

Results in: null

Request with headers:

curl \
-H "Host: app.myallies.com" \
-H "User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:33.0) Gecko/20100101 Firefox/33.0" \
-H "Accept: text/html,application/xhtml,application/xml;q=0.9,*/*;q=0.8" \
-H "Accept-Language: en-US,en;q=0.5" \
-H "Accept-Encoding: gzip, deflate" \
-H "Connection: keep-alive" \
-H "Cache-Control: max-age=0" \
http://app.myallies.com/api/quote/goog`

Results in: {"StockID":0,"LastTradePriceOnly":"496.38","ChangePercent":"0.78"}

You're probably going to have to set some request headers to more closely mimic the request a browser makes.

Ok, the answer makes sense, but if that's the case then how come the request from the exact same code works well with http://app.myallies.com/api/recentnews ? This is what actually dazzles me! — Maya, Jan 13 '15 at 20:38
That is quite weird. Probably a bug on their implementation of those services. 99% of the time the developer would just the front end with a browser, so they would never have noticed the bug. — Jimmy James, Jan 13 '15 at 20:51

JSON Serialisation from URL always returning NULL

2 Answers2