3

How do I create directories in Python given a list of files with paths that may or may not exist?

I am downloading some files from s3 that may or may not exist in a certain directory. Based upon these keys that represent a potentially deeply nested directory that may or may not exist.

So based upon the key /a/number/of/nested/dirs/file.txt how can I created /a/number/of/nested/dirs/ if they do not exist and do it in a way that doesn't take forever to check for each file in the list.

I am doing this because if the local parent directories do not already exist get_contents_to_filename breaks.

My Final Solution Using Answer: 

for file_with_path in files_with_paths:
    try:
        if not os.path.exists(file_with_path):
            os.makedirs(file_with_path)
        site_object.get_contents_to_filename(file_with_path)
    except:
        pass
Mike H-R
  • 7,726
  • 5
  • 43
  • 65
dkroy
  • 2,020
  • 2
  • 21
  • 39
  • @AdamSmith I was about to say that may do it for me, but os.makedirs gets me even further. – dkroy May 08 '14 at 21:32
  • I'm dumb. I read "How do I create DICTIONARIES" not "how do I create directories." – Adam Smith May 08 '14 at 21:33
  • possible duplicate of [mkdir -p functionality in python](http://stackoverflow.com/questions/600268/mkdir-p-functionality-in-python) – Lukas Graf May 08 '14 at 21:35
  • @LukasGraf Probably, thanks for pointing that out, although I feel like mine was a better question. – dkroy May 08 '14 at 21:49
  • 1
    About: `if os.path.exists(file_with_path) is False`. 'is' tests if two variables point to the same object (reference equality). Although it will technically work, using `if not os.path.exists(file_with_path)` is arguably cleaner style-wise. – ChristopheD May 08 '14 at 21:52
  • 1
    @ChristopheD ...but shouldn't be used anyway because it introduces a [race condition](http://stackoverflow.com/questions/14574518/how-does-using-the-try-statement-avoid-a-race-condition). – Lukas Graf May 08 '14 at 21:59
  • 1
    @Lukas Graf. I agree. Just catching the exception would be more pythonic too (in the 'easier to ask forgiveness then permission' sense). – ChristopheD May 08 '14 at 22:01

1 Answers1

5

Simply use os.makedirs(). This will create all intermediate directories if needed.

Recursive directory creation function. Like mkdir(), but makes all intermediate-level directories needed to contain the leaf directory. Raises an error exception if the leaf directory already exists or cannot be created. The default mode is 0777 (octal). On some systems, mode is ignored. Where it is used, the current umask value is first masked out.

ChristopheD
  • 112,638
  • 29
  • 165
  • 179
  • @dkroy Almost. You also need to do some exception handling to get the [functionality of `mkdir -p`](http://stackoverflow.com/questions/600268/mkdir-p-functionality-in-python). – Lukas Graf May 08 '14 at 21:34