How to make multiple file from different folder same name in one file in python

Question

I want to combine multiple file from different folder data in one file but only same file name is all folder

Script:

import os

filenames = [os.path.join('C:/Users/Vishnu/Desktop/Test_folder/Input/','*.txt'), os.path.join('C:/Users/Vishnu/Desktop/Test_folder/Output/','*.txt')]
f = open(r'C:/Users/Vishnu/Desktop/Test_output/', 'wb')
for fname in filenames:
    with open(fname) as infile:
        for line in infile:
            f.write(line)

Getting Error:

f = open(r"C:/Users/Vishnu/Desktop/Test_output/", "wb")
IOError: [Errno 13] Permission denied: 'C:/Users/Vishnu/Desktop/Test_output/'
>>>

After Modification

import os
import glob 
import os.mkdir

filenames = [glob.globos(mkdir(os.path.join(os.path.expanduser('~'),'C:', 'Users' , 'Vishnu' ,'Desktop','Test_folder','Input','*.txt'))), glob.glob(os.mkdir(os.path.join(os.path.expanduser('~'),'C:', 'Users' , 'Vishnu' , 'Desktop','Test_folder','Output','*.txt')))]    
filenames[0].extend(filenames[1])
filenames=filenames[0]

if( not os.path.isdir(os.path.join(os.path.expanduser('~'),'C:', 'Users' , 'Vishnu' , 'Desktop' ,'Test_folder', 'Test_output'))):
    os.mkdir(os.path.join(os.path.expanduser('~'),'C:', 'Users' , 'Vishnu' , 'Desktop' ,'Test_folder', 'Test_output'))
for fname in filenames:
    with open(fname) as file:
        for line in file.readlines():
            f = open(os.mkdir(os.path.join(os.path.expanduser('~'), 'Desktop', 'Test_output')),'{:}.txt'.format(os.path.split(fname)[-1] ), 'a+')
            f.write(line)
            f.close()

Possible duplicate of [Python concatenate text files](http://stackoverflow.com/questions/13613336/python-concatenate-text-files) — CDspace, Oct 19 '16 at 16:45
You are trying to open a _directory_ not a file. `open` only works on files, which is why you are getting that error. — Burhan Khalid, Oct 20 '16 at 06:25
@Burhan Khalid, How to merge three folder files in separate folder? I have three folder different data but all file name is same — krish, Oct 20 '16 at 06:29

R. S. Nikhil Krishna · Answer 1 · 2016-10-20T14:13:39.030

1

Firstly, you are trying to open the folder itself. Secondly, we have to close the file everytime we read it to avoid Permission issues

I tried this code. It should work now

import os
import glob    #So that * in directory listing can be interpretted as all filenames

filenames = [glob.glob(os.path.join(os.path.expanduser('~'),'Desktop','Test_folder','Input','*.txt')), glob.glob(os.path.join(os.path.expanduser('~'),'Desktop','Test_folder','Output','*.txt'))]    
filenames[0].extend(filenames[1])
filenames=filenames[0]

if( not os.path.isdir(os.path.join(os.path.expanduser('~'), 'Desktop', 'Test_output'))):
    os.mkdir(os.path.join(os.path.expanduser('~'), 'Desktop', 'Test_output'))
for fname in filenames:
    with open(fname) as file:
        for line in file.readlines():
            f = open(os.path.join(os.path.expanduser('~'), 'Desktop', 'Test_output','{:}.txt'.format(os.path.split(fname)[-1] )), 'a+')
            f.write(line)
            f.close()    #This should take care of the permissions issue

edited Oct 20 '16 at 14:13

answered Oct 19 '16 at 16:44

R. S. Nikhil Krishna

3,962
1
13
27

I have used above code but getting error, please check my question. I have edited my question – krish Oct 20 '16 at 06:21
Error: f = open(r'C:/Users/Vishnu/Desktop/Test_output/', 'r+') IOError: [Errno 13] Permission denied: 'C:/Users/Vishnu/Desktop/Test_output/' , I want to take two input file from other directories but same name and save to other directories same file name – krish Oct 20 '16 at 10:21
@krish try now and let me know – R. S. Nikhil Krishna Oct 20 '16 at 12:27
Getting error: f.write('{:}\n'.format(line)) ^ SyntaxError: invalid syntax >>> – krish Oct 20 '16 at 13:51
@krish I tried this code. works for me! Hope it helps. – R. S. Nikhil Krishna Oct 20 '16 at 14:13
Still showing error: Traceback (most recent call last): File "C:/Users/Vishnu/Documents/NAD/NAD/result/test_result_file/Test_9.py", line 9, in os.mkdir(os.path.join(os.path.expanduser('~'), 'Desktop', 'Test_output')) FileNotFoundError: [WinError – krish Oct 20 '16 at 14:33
I have tried python 3.5 and 2.7 also but error is same. How to fix it? – krish Oct 20 '16 at 14:33
When you try just `print(filenames)`, what does it show? – R. S. Nikhil Krishna Oct 20 '16 at 15:19
os.mkdir(os.path.join(os.path.expanduser('~'), 'Desktop' ,'Test_folder', 'Test_output')) WindowsError: [Error 3] The system cannot find the path specified: 'C:\\HOME\\Desktop\\Test_folder\\Test_output' – krish Oct 20 '16 at 15:51
Still getting error: ''' os.mkdir(os.path.join(os.path.expanduser('~'), 'Desktop', 'Test_output')) WindowsError: [Error 3] The system cannot find the path specified: 'C:\\HOME\\Desktop\\Test_output' ''' >>> – krish Oct 22 '16 at 06:51
Try replacing `os.mkdir(os.path.join(os.path.expanduser('~'), 'Desktop', 'Test_output'))` with `os.mkdir('C:\\Users\\Vishnu\\Desktop\\Test_output')` – R. S. Nikhil Krishna Oct 22 '16 at 09:43
I tried but mot getting output? I have edited my question above with updated script. Can you check once? – krish Oct 22 '16 at 09:57

score 0 · Answer 2 · answered Oct 20 '16 at 06:35

As I understand it, you have multiple files in multiple directories. Some of these files have the same name, and you want to combine those files with the same name as one file, in another directory.

The first task, is to figure out what are all the unique file names in the directories; and collect the paths to those unique files.

import os, glob
from collections import defaultdict

dirs = [r'/foo/bar/directory_1', r'/foo/bar/directory_2']
file_pattrn = r'*.txt'
unique_files = defaultdict(list)

for d in dirs:
  for i in glob.iglob(os.path.join(d, file_pattrn)):
     unique_files[os.path.basename(i)].append(i)

Now we have a dictionary, where the key is the filename, and the value is a list of all the paths to that file.

Once we have that, then we just loop through them, to create our new files:

destination = r'/foo/bar/new_directory'
for unique_filename, copies in unique_files.items():
   with open(os.path.join(destination, unique_filename), 'w') as f:
     for copy in copies:
        with open(copy, 'r') as cp:
           for line in cp:
              f.write(line)

Thanks, I have used your code, but getting only first folder data? — krish, Oct 20 '16 at 07:55

How to make multiple file from different folder same name in one file in python

2 Answers2