Python program reading URL

Question

I am looking for a python program which will run the URL. Simple, it just needs to run the URL produced from the application with provided credentials to access the application. I will schedule to run the python script every night.

I have an application that produces the URL. If I run that URL, it will produce the JSON file.

I am trying to pre-cache the output of the application so I want to run the URL every morning.

Below are the information:

USERNAME = "test"
PASSWORD = "test5"
HOST = 'appl.xyz.net' 
PORT = 8080

Sample URL is: http://appl.xyz.net:8080/app/content/pq/doQuery?solution=nd&path=&file=Test.nd&dataAccessId=1&paramid=4221

JSON:

{
   "queryInfo":{
      "totalRows":"3"
   },
   "resultset":[
      [
         4215,
         null,
         "AAA"
      ],
      [
         4215,
         null,
         "BBB"
      ]
   ],
   "metadata":[
      {
         "colIndex":0,
         "colType":"Numeric",
         "colName":"id"
      },
      {
         "colIndex":1,
         "colType":"String",
         "colName":"Name"
      },
      {
         "colIndex":2,
         "colType":"String",
         "colName":"City"
      }
   ]
}

Thanks

score 1 · Accepted Answer · edited May 23 '17 at 10:31

1

Use the python-requests library.

Then all you need to do is:

import requests

url = 'http://appl.xyz.net:8080/app/content/pq/doQuery?solution=nd&path=&file=Test.nd&dataAccessId=1&paramid=4221'
user = 'test'
password = 'test5'

# Takes care of the HTTP authentication
data = requests.get(url, auth=(user, password))
json_data = data.json()

StvnW has provided a link in the comments to CURL alternative in Python if you do not wish to install any other libraries.

edited May 23 '17 at 10:31

Community

1
1

answered Oct 17 '13 at 16:03

Ewan

14,592
6
48
62

where should I enter credentials? – Rio Oct 17 '13 at 16:04
1

Read the `requests` documentation. But I've updated my answer to include the HTTP authentication. – Ewan Oct 17 '13 at 16:06
1

More related discussion and examples, including comparisons between `requests` and `urllib2` : http://stackoverflow.com/questions/2667509/curl-alternative-in-python/8131577#8131577 http://stackoverflow.com/questions/2018026/should-i-use-urllib-or-urllib2-or-requests – StvnW Oct 17 '13 at 16:08
Can you show me how to do without installing library (e.g. using import urllib2) because I could not install it in windows.I did "setup.py install" in cmd prompt. – Rio Oct 17 '13 at 17:33
Try it yourself using the link @StvnW posted in the comments and me in the question. Also look at using `easy_install` or `pip` on Windows to help install packages. Also what was the output of `setup.py install`? Try and work out why it's not installing. – Ewan Oct 17 '13 at 17:49
Another way to install (many, but by no means all) packages on Windows is to go to [Christoph Gohlke's site](http://www.lfd.uci.edu/~gohlke/pythonlibs/) and see if he has a binary installer. For `requests`, he does. So, you can just download the one that matches your Python version and run it. – abarnert Oct 17 '13 at 17:59
can you show code how to do using urllib2 in python 2.7? thanks – Rio Oct 17 '13 at 17:59
I was able to use install library. Thanks! My login are: "User Name:" "assword:" When I run the script it runs for 10 mins and times out. Actually it runs for 20 secs. Is it because the login box with different name? – Rio Oct 17 '13 at 19:03

Python program reading URL

1 Answers1