Generating file and .write is happening AFTER .call even though not written that way

Question

I'm trying to pass a java file via python and write the shell output into a log .txt file. This is working except that my manual breaks are being added to the end of the file. Any advice on why .write is happening after subprocess.call? I'm very new to Python. Thanks

Code Snippet

import subprocess

outfile = open('c:\\log2.txt', 'a')
outfile.write("\n\n")
outfile.write("-------------------------BEGIN NEW LOG--------------------------------------")
outfile.write("\n\n")
subprocess.call(['java.exe',
                 '-cp',
                 'c:\\dev\\dataloader\\dataloader-36.0.0-uber.jar',
                 '-Dsalesforce.config.dir=c:\\dev\\dataloader\\',
                 'com.salesforce.dataloader.process.ProcessRunner',
                 'process.name=csvCRExtract'],
                stdout=outfile)
outfile.close()

What happens if you call `close()` before you call `call()`? — Kevin, Apr 03 '17 at 19:35
you probably want to call `outfile.flush()` before the `subprocess.call()`. It may be a race condition for which process wins in emptying their output buffers. — Harvey, Apr 03 '17 at 19:35
@Kevin, that didn't work because I was closing the file (I assume). Here's the error: `ValueError: I/O operation on closed file` — Grant, Apr 04 '17 at 00:29
@Harvey that did work, but can you define "race" please? Will this not always return the same result? Thanks! — Grant, Apr 04 '17 at 00:31

score 0 · Accepted Answer · edited May 23 '17 at 12:10

Call output.flush() before subprocess.call().

output is using buffered i/o, meaning that it waits until a minimum amount of data has been written before actually writing the data to the operating system to avoid lots of slow system write calls. flush tells the buffer to send everything to the OS immediately. Your program was doing the following:

writing your log header to the local buffer
calling your java process which writes to its own buffer and then to the file when the process exits
flushing your local buffer to the file when you close the file.

This answer has a picture and explains it a bit more:

https://stackoverflow.com/a/1450563/47078

Generating file and .write is happening AFTER .call even though not written that way

1 Answers1