Python: work with recursive folders to read and write

Question

I have this code:

# cwd = "C:\Users\johnr\Desktop\myFolder" - current working directory
for filename in os.listdir(os.path.join(cwd, "content")):
    header_file = open(header_file_dir, "r")
    footer_file = open(footer_file_dir, "r")
    if ".md" in filename:
        newFilename = filename.replace(".md", ".html")
    if ".tile" in filename:
        newFilename = filename.replace(".tile", ".html")
    elif ".html" in filename:
        newFilename = filename
    elif ".txt" in filename:
        newFilename = filename.replace(".txt", ".html")
    else:
        print(filename+" is not a valid file type!")

    currents_working_file = open(os.path.join(cwd, "build", newFilename), "w")

    # Write the header
    currents_working_file.write(header_file.read())

    # Get the actual stuff we want to put on the page
    text_content = open(os.path.join(cwd, "content", filename), "r")
    if ".md" in filename:
        text_cont1 = "\n"+markdown.markdown(text_content.read())+"\n"
    elif ".tile" in filename:
        text_cont1 = "\n"+textile.textile(text_content.read())+"\n"
    elif ".html" in filename:
        text_cont1 = text_content.read()
    elif ".txt" in filename:
        text_cont1 = text_content.read()
    else:
        print(filename+" is not a valid file type!")

    # Write the text content into the content template and onto the build file
    content_templ_dir = os.path.join(cwd, "templates", "content_page.html")
    if os.path.exists(content_templ_dir):
        content_templ_file = open(content_templ_dir, "r")
        content_templ_file1 = content_templ_file.read()
        content_templ_file2 = content_templ_file1.replace("{page_content}", text_cont1)
        currents_working_file.write(content_templ_file2)
    else:
        currents_working_file.write(text_cont1)

    # Write the footer to the build file
    currents_working_file.write("\n"+footer_file.read())

    # Close the build file
    currents_working_file.close()

which searches for a file in the 'content' directory and then creates a file of the same name in the'build' directory. How can I make this work when there are files in folders in the 'content' directory?

It seems like this code will create a file in the build directory with the name of any directories found in the content directory. — Joshua Grigonis, Dec 19 '16 at 20:26
Possible duplicate of [How to identify whether a file is normal file or directory using python](http://stackoverflow.com/questions/955941/how-to-identify-whether-a-file-is-normal-file-or-directory-using-python) — , Dec 19 '16 at 20:26
Sounds like you want `os.walk` for walking through files at multiple levels of a directory tree. — jez, Dec 19 '16 at 20:30

score 0 · Answer 1 · answered Dec 19 '16 at 20:47

0

In order to recursively traverse directories, Python provides os.walk:

for root, dirs, files in os.walk(os.path.join(cwd, "content")):
    relative_path = os.path.relpath(root, os.path.join(cwd, "content"))
    for filename in files:
        currents_working_file = open(os.path.join(cwd, "build", relative_path, filename), "w")

answered Dec 19 '16 at 20:47

André Sassi

1,076
10
15

This will fail with `FileNotFoundError: [Errno 2] No such file or directory` if the corresponding directories to the subdirs of `content/` which contain the files don't already exist under `build/`. – das-g Dec 19 '16 at 20:59

score 0 · Answer 2 · answered Dec 20 '16 at 00:11

0

Assuming that cwd just holds the path to the current working dir:

from pathlib import Path
from itertools import chain

source_extensions = {'md', 'html', 'txt'}
source_root_dir_path = Path("content")
source_file_paths = chain.from_iterable(
    source_root_dir_path.glob("**/*.{}".format(ext)) for ext in source_extensions
)
for p in source_file_paths:
    destination_file_path = Path("build", *p.with_suffix(".html").parts[1:])
    destination_file_path.parent.mkdir(parents=True, exist_ok=True)
    with destination_file_path.open('w') as f:
        f.write(header_file.read())
        f.write("\n")
        f.write(footer_file.read())

answered Dec 20 '16 at 00:11

das-g

9,718
4
38
80

So sorry, but with my other code it didn't work so I had to post all of my code. Sorry, would you update please? – John Roper Dec 20 '16 at 00:47
Please don't arbitrarily change the question thereby invalidating existing answers. If you have problems getting your additional code to work with the approach of this answer, that might be a case for a new, separate question. (Feel free to link it here in a comment, if you post it as such.) However, SO isn't a code-writing service, so please take the time to reduce the new problem to a [mcve]. – das-g Dec 20 '16 at 12:11

Python: work with recursive folders to read and write

2 Answers2