Hi, I've been trying to do a retrieval in python

Question

I've been trying to do a retrieval in python

You don't have access to the `/tmp/` directory (odd). Try running Python with `sudo`. — Blender, Nov 14 '11 at 16:56
@Blender: the script has access to `/tmp` otherwise there would be no `download_file(item)` in the traceback. It would fail sooner. — jfs, Nov 14 '11 at 16:59
@J.F.Sebastian: What do you mean? The traceback points to the first line that accesses `/tmp/`. — Blender, Nov 14 '11 at 17:16
@J.F.Sebastian: Oh, I completely missed that recursive call (facepalm) — Blender, Nov 14 '11 at 17:31
I figured out that I have no access to Footemp as it is owned by another user. So i had to create Attretrieve instead of Footemp and own that myself. Apparently, it works for the other user but still doesn't work for me. :( — Doomer, Nov 14 '11 at 21:54
Also, the idea is to read from the list of file name, and retrieve each file into a target directory. I am not sure how much disk space the documents will take up; so we might want to modify the program to retrieve the files just to check the size, but not to store them. — Doomer, Nov 14 '11 at 21:56

score 2 · Answer 1 · edited May 23 '17 at 12:03

the second argument to urlretrieve() is a filename not a directory.
you're ignoring docid argument
you probably don't want to call download_file() recursively
`folder_size` (with backtick) is a syntax error

Here's a script that downloads documents specified in /tmp/docids to /tmp/Footemp directory (it must exist):

outdir = "/tmp/Footemp"
with open("/tmp/docids") as f:
     for line in f:
         docid = line.strip()
         if not docid: continue # skip empty lines
         name = urllib.parse.quote_plus(docid)
         url = "http://foo.com/doc/" + name
         path = os.path.join(outdir, name)
         name, info = urllib.request.urlretrieve(url, path)
         print(info)

See Calculating a directory size using Python?

+1 from me. I was just writing up a similar answer ;) – Blender Nov 14 '11 at 17:34 — Blender, Nov 14 '11 at 17:34

score 0 · Answer 2 · answered Nov 14 '11 at 17:13

0

The code looks to be correct, so it is likely a permissions error. Try saving to a file in a directory that you know you have write access to.

answered Nov 14 '11 at 17:13

Raymond Hettinger

216,523
63
388
485

Hi, I've been trying to do a retrieval in python

2 Answers2