0

I'm using requests get, and fairly often an exception is raised (json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)). I want to loop until it's successful, and not discriminate on the exception type (yet). My code will either succeed first time (exit the loop) or keep looping indefinitely.

def loop_json(self,url):
    result = None
    print(url)
    while result == None:
        try:
            print('trying')
            response = requests.get(url).json()
            result = 1
        except:
            time.sleep(3)
            pass
    return response
halfwaycrook
  • 1
  • 1
  • 2
    400 is a "bad request". It means your request is incorrect in some way, so no matter how many times you batter the server with that request, it will never be valid. – roganjosh Nov 16 '17 at 02:16
  • As @roganjosh said, here is more explanation https://stackoverflow.com/questions/19671317/400-bad-request-http-error-code-meaning – Nabin Nov 16 '17 at 02:19
  • Your code is looping on error for me. I am just calling loop_json as a function, not as part of a class (so I removed self). `In [61]: loop_json(0,"asdf") asdf trying trying trying trying trying trying trying ` – Martin Nov 16 '17 at 02:20
  • @bigmacd the issue is not that it won't loop again on error, it's that the request can never be valid no matter how many times you retry the request, so it's illogical to keep trying the same thing. – roganjosh Nov 16 '17 at 02:24
  • @roganjosh - I've linked to two pics: one it succeeds, the other it doesn't. It's the same code as above, literally just ran it twice, it looped infinitely the first time and succeeded the second. Quite sure it isn't an incorrect request 1 - https://imgur.com/a/ZSJFK 2 - https://imgur.com/a/99ily – halfwaycrook Nov 16 '17 at 03:29
  • You're quite sure it's not an incorrect request except the error code 400 is _specifically_ associated with a malformed request and the infinite loop for that request indicates that it will never succeed? The URL for one of your requests is malformed. How else do you explain the behaviour? – roganjosh Nov 16 '17 at 03:31
  • So it's possible for the incorrect request to sometimes return data anyway? It's the same request in both pics. I mean I know 100% that the URL I'm sending is correct, whether there's something with my code that's the incorrect way to request something, I'm not sure – halfwaycrook Nov 16 '17 at 03:34
  • Ok, I don't remember the pics being in your comment when you initially wrote it so I wasn't sure what you're referring to. Ok, by infinite loop, how long have you left it to run for? – roganjosh Nov 16 '17 at 03:38
  • 1
    My mistake, it isn't a 400 error, I must have been printed that error when I actually was sending an incorrect request. Will edit OP now – halfwaycrook Nov 16 '17 at 03:38
  • I've left it to run for 20+ loops a few times, which I assume is enough to conclude it's getting hung up. It has not once failed once then succeeded in under 5 tries in the many times I've run it – halfwaycrook Nov 16 '17 at 03:39
  • 400 was quite misleading there. I wonder if it's rate-limited; on `except` try `response = requests.get(url)` and `print(response)` and `print(response.text)`. That may or may not break your exception handler, but it could also be that the server is giving a non-json response explaining the problem. Worth a try. – roganjosh Nov 16 '17 at 03:45
  • Interesting, it does return a 400 response. To be clear, I just ran it 4 separate times - the first 3 returned everything well (response 200) and the 4th time, with nothing changed, your suggested print statements return: – halfwaycrook Nov 16 '17 at 08:46
  • Posted it in new 'answer' so this thread doesn't get too long – halfwaycrook Nov 16 '17 at 08:49
  • 1
    Posting as an answer is a bad idea. Your 'answer' doesn't answer the question but it now appears to others that the question is answered. That's not how this site works. – roganjosh Nov 16 '17 at 18:05

1 Answers1

0

I'll just write this here so it doesn't become a really long comment thread - it is indeed a 400 error that's returned. I've posted an imgur url with the response I get. This only happens sometimes - I ran it 4 times, the first 3 times I got a 200 response and everything was fine, the 4th I got a 400 response. Nothing changed, and the loop still stays looping once I get the 400

https://i.stack.imgur.com/ALjhN.jpg

halfwaycrook
  • 1
  • 1