Reading text file from different directories with python os.walk

Question

I have 5 directories and 25 text files in each directory. When I try to read text files I got an error msg like '[Errno 2] No such file or directory: 'AEITMYIRQLP.txt'

My codes:

import re
import os
for roots,dirs,files in os.walk(spath):
    for file in files:
        with open(file,'r') as f:
            readf=f.readlines()
            for line in readf:
                lstNumbers=[]
                pattern =re.compile(r'\d{3}-\d{3}-\d{4}')
                matches=re.findall(pattern,line)
                for match in matches:
                    lstNumbers.extend(match)

I think the code is looking for AEITMYIRQLP.txt in the current directory. But note that the files returned as the 3rd item from `os.walk` are just the file names. You need to join it with the directory they are in, which is assigned to root, the first item returned from `os.walk`. [Here](https://stackoverflow.com/a/19308592/245915) is an example. — nofinator, Mar 16 '21 at 02:57

score 0 · Answer 1 · answered Mar 16 '21 at 03:15

As @nofinator astutely mentions in a comment, os.walk() needs to be used carefully to ensure the full paths are produced.

There were a couple of other issues with your code involving lstNumbers, and what you were doing to it.

Here's a way to fix your code using the newer pathlib to join what's in root to the filenames, and addressing the lstNumbers issues:

import pathlib

lstNumbers = [] # declare it here so you don't overwrite it for each file
pattern = re.compile(r'\d{3}-\d{3}-\d{4}') # declare here to avoid re-declaring

for roots,dirs,files in os.walk(spath):
    for fl in files: # changed to `fl` so we don't shadow built-in `file`
        with open(pathlib.Path(roots).joinpath(fl),'r') as f:
            for line in f:
                lstNumbers = []
                matches = re.findall(pattern,line)
                for match in matches:
                    lstNumbers.append(match) # `.append()` instead of `.extend()`

Reading text file from different directories with python os.walk

1 Answers1