python subprocesses are not terminating

Question

To execute various tools on a set of files, I use the following Command class to call them.

import subprocess
import threading
import logging
logger = logging.getLogger('root')


class Command(object):
    def __init__(self, cmd):
        self.cmd = cmd
        self.process = None

    def run(self, timeout, logfile):
        def target():
            logger.info('Thread started')
            logger.info('Command: %s' % self.cmd)
            if logfile is None:
                self.process = subprocess.Popen(self.cmd, shell=True)
            else:
                logger.info('logging to file %s' % logfile.name)
                self.process = subprocess.Popen(self.cmd, shell=True, stdout=logfile)
            self.process.communicate()
            logger.info('Thread finished')
            self.process.kill()
        thread = threading.Thread(target=target)
        # make it a daemon
        thread.daemon = True
        thread.start()
        thread.join(timeout)
        if thread.is_alive():
            logger.warn('Terminating process')
            thread.join()
            self.process.kill()
        logger.info('Thread Returncode: %d' % self.process.returncode)
        return self.process.returncode

The problem I face is:

that the tools/commands I run do not Terminate properly, especially if the python program runs long (3+ hours).

score 1 · Answer 1 · answered Sep 29 '17 at 07:51

1

The problem is that you're opening the processes with shell=True

self.process = subprocess.Popen(self.cmd, shell=True, stdout=logfile)

Solution here: How to terminate a python subprocess launched with shell=True

answered Sep 29 '17 at 07:51

ConorSheehan1

1,640
1
18
30

So I replace self.process.kill() with the solution from your link I guess. – Sep 29 '17 at 07:57
I have a very similar problem (also using subprocess with shell=True) but what I would like to do is to simply wait until the process (ping cmd with --count flag) will terminate just by itself - that is I cannot send the kill() signal. Any suggestions how it may be obtained? Thx!! – Marek Bocian Feb 06 '21 at 16:55

python subprocesses are not terminating

1 Answers1