csv_readout folder in wrong series

Question

I want to read out data from different files in one folder. The files have the names: "1.csv", "2.csv", "3.csv" ... "96.csv". But instead of reading them in from the top to the bottom, it reads in "1.csv", "10.csv", "11.csv"... "2.csv", "21.csv". Anyone knows how to fix this problem?

Thanks!

def csv_readout_folder(path):    
    os.chdir(path)     
    files = glob.glob(path +'/'+'*.csv')
    all_data = pd.DataFrame()

    for f in files:
        data = csv_readout(path,f)
    
        all_data = pd.concat([all_data, data])
           
    return all_data

Does this answer your question? https://stackoverflow.com/questions/6773584/how-is-pythons-glob-glob-ordered — match, Sep 28 '21 at 13:00
Does this answer your question? [Sort filenames in directory in ascending order](https://stackoverflow.com/questions/33159106/sort-filenames-in-directory-in-ascending-order) — w-m, Sep 28 '21 at 13:33

Virtuoz · Answer 1 · 2021-09-28T13:18:58.640

0

You can do something like

files = [f'{path}/{i}.csv' for i in range(1, 22)]

instead of

files = glob.glob(path +'/'+'*.csv')

UPD:

def csv_readout_folder(path):    
    os.chdir(path)
    n_files = len([el for el in os.scandir(path) if el.is_file()]) 
    files = [f'{path}/{i}.csv' for i in range(1, n_files + 1)]
    all_data = pd.DataFrame()

    for f in files:
        data = csv_readout(path,f)
    
        all_data = pd.concat([all_data, data])
           
    return all_data

edited Sep 28 '21 at 13:18

answered Sep 28 '21 at 13:04

Virtuoz

896
1
8
14

Yes that works, the only problem is, that the total number of csv files in the folder changes. Do you know how I can fix this? So that I can use the code for a varying number of files? – Isabel Waibel Sep 28 '21 at 13:13
Pass this total number of files as an argument to your function – Virtuoz Sep 28 '21 at 13:14
And how do I get the total number of files in the folder? – Isabel Waibel Sep 28 '21 at 13:15
This can be done via, for example len([el for el in os.scandir(path) if el.is_file()]) – Virtuoz Sep 28 '21 at 13:17
@IsabelWaibel I added a bit more explanation to the answer. Please, check if it works for you now – Virtuoz Sep 28 '21 at 13:19

score 0 · Answer 2 · answered Sep 28 '21 at 13:15

In you code for f in files: should read the files in the order they appear in the list. You can try sort functions but it may be easier to make a new list like this:

file_lst=[]
for k in range(1,97):
    file_lst.append(f'{str(k)}.csv')

s1=pd.Series(file_lst)

def csv_readout_folder(path):    
    os.chdir(path)     
    files = glob.glob(path +'/'+'*.csv')
    all_data = pd.DataFrame()

    for f in list(s1[s1.isin(file_lst)]):
        data = csv_readout(path,f)
    
        all_data = pd.concat([all_data, data])
           
    return all_data

csv_readout folder in wrong series

2 Answers2