Concatenate files content in one file using python

Question

I have in a directory 'C:/Users/lamda/Desktop/ML working/logs/a/' multiple sub-directories and each sub-directory contains a bunch of files.my goal is to concatenate all these files in one file 'C:/Users/lamda/Desktop/ML working/logs/concatenate.log' I have wrote the following python code but it only succeeds in copying one file content in 'C:/Users/lamda/Desktop/ML working/logs/concatenate.log'.I'm doubting that there a missing for loop but could not really figure out the issue.Please help me

import glob
import os.path
directories = os.listdir('C:/Users/lamda/Desktop/ML working/logs/a')
for i in directories:
    files = os.listdir('C:/Users/lamda/Desktop/ML working/logs/a/' + i)
    print(files[1:])
    files2 = files[1:]
    for j in files2 :       
        fs = open('C:/Users/lamda/Desktop/ML working/logs/a/'+i+'/'+j,'r')
        fd = open('C:/Users/lamda/Desktop/ML working/logs/concatenate.log','w')
        #print(i)
        #print(j)
        for ligne in fs :
            fd.write(ligne)
fd.close()
fs.close()

Please can you correct your indentation. This is critical to python. In particular, your for loop `for i in directories:` has no body. — Martin Bonner supports Monica, Apr 03 '19 at 09:37

score 2 · Accepted Answer · answered Apr 03 '19 at 09:33

This is because when you open(path, 'w') the file is truncated. You either need to open the target file just once and use the file-like object in all iterations of the loop over source files otherwise you have to open the file in append mode open(path, 'a').

See https://docs.python.org/2/library/functions.html#open for details.

score 0 · Answer 2 · answered Apr 03 '19 at 09:32

Something like this will work.

OF = open("out.txt", 'w+')
for path, dirlist, name in os.walk(path):
    fpath = (os.path.join(path, n) for n in name)
    try:
        for file in fpath:
            with open(file, "r") as f:
                for line in f:
                    OF.write(line)
    except Exception as e:
        print(e)

Concatenate files content in one file using python

2 Answers2