how to write a list into a txt file in python 3?

Question

I want to make a program (Python 3.6) that can read multiple log files and write all the lines from those files into a txt file.

The code that I've already tried can read all the lines, but can't write all lines to a txt file. I've tried this:

allFiles = glob.glob('C:\Program Files\PostgreSQL\9.6\data\pg_log\*.log')
def readFile(allFiles):
    for file in allFiles:
        f = open(file,'r')
        allLines = []
        for line in f:
            allLines.append(line)
            print(line)
        f.close()

    with open ('readFile.txt',mode='wt', encoding='utf-8') as fileOutput:
        for line in allLines:
            fileOutput.write(line)
        fileoutput.close()

I expect all the lines from all files can written in a txt file, but results I've got was lines written in txt only line that have the same date as the date of execution of this program.

What should I do ?

GreenMatt · Accepted Answer · 2019-02-19T16:18:52.153

You are resetting allLines to an empty list each time you read a file. Move the allLines = [] line to outside your for loop. In other words, the start of your function should be:

def readFile(allFiles):
    allLines = []
    for file in allFiles:
        f = open(file,'r')

Also, a couple "style notes": It is generally considered bad form to use a variable name that already has meaning in Python, even though the language allows it. Thus it would be a good idea to use something other than "file". Also, Python convention is that underscores are used between words in variable names, not Camel Case. Thus "allLines" would be "all_lines". You might want to see the Python Style Guide at: https://www.python.org/dev/peps/pep-0008/

Patrick Artner · Answer 2 · 2019-02-19T16:24:30.660

alllines only holds the lines from the last read file - because you reset it inside the loop to alllines=[]. You need to move this before the loop (or get rid of it - see below).

It would be far easiert to simply concat the files: see Python concatenate text files

Your code has a second/third problem - if you have 20 files that each have 1 GB you store 20GB ... in memory ... which is highly inefficient.

It would be better to just write line by line into your new file. Beside that your should use the with open(...) as ..: paradigma when using file objcts like so:

def readFile(allFiles):
    with open ('readFile.txt', mode='wt', encoding='utf-8') as fileOutput:
        for file in allFiles:
            with open(file) as reader:
                for line in reader:
                    fileOutput.write(line)


allFiles = glob.glob('C:\Program Files\PostgreSQL\9.6\data\pg_log\*.log')
readFile(allFiles)

See Python 3.7 - File reading/writing

how to write a list into a txt file in python 3?

2 Answers2