1
url = "www.someurl.com"

request = urllib2.Request(url,header={"User-agent" : "Mozilla/5.0"})

contentString = urllib2.url(request).read()

contentFile = StringIO.StringIO(contentString)

for i in range(0,2):
    html = contentFile.readline()

print html

The above code runs fine from commandline but if i add it to a cron job it throws the following error:

  File "/usr/lib64/python2.6/urllib2.py", line 409, in _open
    '_open', req)
  File "/usr/lib64/python2.6/urllib2.py", line 369, in _call_chain
    result = func(*args)
  File "/usr/lib64/python2.6/urllib2.py", line 1186, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib64/python2.6/urllib2.py", line 1161, in do_open
    raise URLError(err)
urllib2.URLError:

I did look at some tips on the other forums and tried it but it has been of no use.

Any help will be much appreciated.

Nischal Hp
  • 435
  • 1
  • 6
  • 21
  • I would check the following: 1. Make sure that the same python version and python path is used in both cases. Insert a ``print sys.version; print sys.path`` at the top of the program and compare 2. Compare environment variables (``print os.environ``). Check if a ``HTTP_PROXY`` environment variable is present. – codeape Feb 12 '13 at 07:23
  • The version is the same and yes HTTP_PROXY is present. is that the problem? – Nischal Hp Feb 12 '13 at 08:35
  • 1
    I bet it is a problem with your proxy settings. URLError: – Kiwisauce Feb 12 '13 at 08:49
  • But the thing that is confusing is on the same machine how does the python script run from commandline? Shouldn the proxy cause a problem even then? or are the Os settings used only after i create a cron job? – Nischal Hp Feb 12 '13 at 09:09
  • 2
    Double check that the HTTP_PROXY environment variable is seen by the cronjob task. Do a ``print os.environ`` from the program and make sure that output is redirected so that you can inspect the value. – codeape Feb 12 '13 at 11:03
  • Thanks a lot. Before my job i add .$HOME/.profile and it works now. – Nischal Hp Feb 13 '13 at 04:51

1 Answers1

1

The environment variables that were used by crontab and from the command line were different.

I fixed this by adding */15 * * * * . $HOME/.profile; /path/to/command.

This made the crontab to pick up enivronment variables that were specified for the system.

Community
  • 1
  • 1
Nischal Hp
  • 435
  • 1
  • 6
  • 21