terminating a function call in python after n seconds

Question

my python code goes like this:

def a():
    ...  
    ...  
    subprocess.call()  
    ...  
    ...  

def b():  
    ...  
    ...

and so on.

My task:
1) If subprocess.call() returns within 3 seconds, my execution should continue the moment subprocess.call() returns.
2) If subprocess.call() does not return within 3 seconds, the subprocess.call() should be terminated and my execution should continue after 3 seconds.
3) Until subprocess.call() returns or 3 seconds finishes, the further execution should not take place.

This can be done with threads but how?

Relevant part of the real code goes like this:

...  
cmd = ["gcc", "-O2", srcname, "-o", execname];    
p = subprocess.Popen(cmd,stderr=errfile)//compiling C program  
...  
...  
inputfile=open(input,'w')  
inputfile.write(scanf_elements)  
inputfile.close()  
inputfile=open(input,'r')  
tempfile=open(temp,'w')
subprocess.call(["./"+execname,str(commandline_argument)],stdin=inputfile,stdout=tempfile); //executing C program
tempfile.close()
inputfile.close()  
...  
...

I am trying to compile and execute a C program using python. When I am executing C program using subprocess.call() and suppose if the C program contains an infinite loop, then the subprocess.call() should be terminated after 3 seconds and the program should continue. I should be able to know whether the subprocess.call() was forcefully terminated or successfully executed so that I can accordingly print the message in the following code.

The back end gcc is of linux.

possible duplicate of [subprocess with timeout](http://stackoverflow.com/questions/1191374/subprocess-with-timeout) — jfs, Jan 18 '15 at 12:44
related: [Stop reading process output in Python without hang?](http://stackoverflow.com/a/4418891/4279) — jfs, Jan 18 '15 at 12:49

score 2 · Answer 1 · edited May 23 '17 at 11:45

2

My task:
1) If subprocess.call() returns within 3 seconds, my execution should continue the moment subprocess.call() returns.
2) If subprocess.call() does not return within 3 seconds, the subprocess.call() should be terminated and my execution should continue after 3 seconds.
3) Until subprocess.call() returns or 3 seconds finishes, the further execution should not take place.

On *nix, you could use signal.alarm()-based solution:

import signal
import subprocess

class Alarm(Exception):
    pass

def alarm_handler(signum, frame):
    raise Alarm

# start process
process = subprocess.Popen(*your_subprocess_call_args)

# set signal handler
signal.signal(signal.SIGALRM, alarm_handler)
signal.alarm(3) # produce SIGALRM in 3 seconds

try:
    process.wait() # wait for the process to finish
    signal.alarm(0) # cancel alarm
except Alarm: # subprocess does not return within 3 seconds
    process.terminate() # terminate subprocess
    process.wait()

Here's a portable threading.Timer()-based solution:

import subprocess
import threading

# start process
process = subprocess.Popen(*your_subprocess_call_args)

# terminate process in 3 seconds
def terminate():
    if process.poll() is None:
        try:
            process.terminate()
        except EnvironmentError:
            pass # ignore 

timer = threading.Timer(3, terminate)
timer.start()
process.wait()
timer.cancel()

edited May 23 '17 at 11:45

Community

1
1

answered Jan 22 '15 at 04:17

jfs

399,953
195
994
1,670

Issue in this is that it might block if process.stdout.readline() doesn't return according to your link. But I want this feature also. – user8109 Jan 22 '15 at 06:45
@user8109: the output is redirected to a file in your question. `process.stdout` is `None` in this case. If you need to read the output then *update your quesiton* and provide the code that you actually use – jfs Jan 22 '15 at 06:47
If the code contains more scanf statements than the content in the file, the C code will be stuck and your code as well. Am I right? – user8109 Jan 22 '15 at 07:02
The code which I have displayed in question is what I actually use. – user8109 Jan 22 '15 at 07:05
the code works as is. Just try it. If the C code is stuck then the alarm goes off and `process.terminate()` is called. If the code does not define SIGTERM handler then the subprocess exits. If the program does not exit on SIGTERM; you could use `process.kill()` that kills almost anything. If the code as shown in the question then `process.stdout` is `None` and you can't call `readline()`. – jfs Jan 22 '15 at 07:07

score 1 · Accepted Answer · answered Jan 22 '15 at 15:48

Finally the below code worked:

import subprocess
import threading
import time


def process_tree_kill(process_pid):
    subprocess.call(['taskkill', '/F', '/T', '/PID', process_pid])

def main():
    cmd = ["gcc", "-O2", "a.c", "-o", "a"];  
    p = subprocess.Popen(cmd)
    p.wait()
    print "Compiled"
    start = time.time()

    process = subprocess.Popen("a",shell=True)
    print(str(process.pid))   

    # terminate process in timeout seconds
    timeout = 3 # seconds
    timer = threading.Timer(timeout, process_tree_kill,[str(process.pid)])
    timer.start()

    process.wait()
    timer.cancel()

    elapsed = (time.time() - start)
    print elapsed

if __name__=="__main__":
    main()

1. you could use `call(cmd)` instead of `Popen(cmd).wait()` 2. drop `shell=True`: (a) provide the full path (directory and the file extension) if Popen can't find the executable (there are different rules how programs are found). (b) `process.terminate` would be enough instead of `kill_process_tree` (no parent shell process, only C child process) — jfs, Jan 22 '15 at 16:07

score 0 · Answer 3 · answered Jan 18 '15 at 12:08

If you're willing to convert your call to a Popen constructor instead of call (same way you are running gcc), then one way to approach this is to wait 3 seconds, poll the subprocess, and then take action based on whether its returncode attribute is still None or not. Consider the following highly contrived example:

import sys
import time
import logging
import subprocess

logging.basicConfig(format='%(asctime)s %(levelname)s %(message)s', level=logging.INFO)

if __name__ == '__main__':
  logging.info('Main context started')
  procCmd = 'sleep %d' % int(sys.argv[1])
  proc = subprocess.Popen(procCmd.split())

  time.sleep(3)
  if proc.poll() is None:
    logging.warning('Child process has not ended yet, terminating now')
    proc.terminate()
  else:
    logging.info('Child process ended normally: return code = %s' % str(proc.returncode))

  logging.info('Main context doing other things now')
  time.sleep(5)
  logging.info('Main context ended')

And this results in different logging output depending upon whether the child process completed within 3 seconds or not:

$ python parent.py 1
2015-01-18 07:00:56,639 INFO Main context started
2015-01-18 07:00:59,645 INFO Child process ended normally: return code = 0
2015-01-18 07:00:59,645 INFO Main context doing other things now
2015-01-18 07:01:04,651 INFO Main context ended
$ python parent.py 10
2015-01-18 07:01:05,951 INFO Main context started
2015-01-18 07:01:08,957 WARNING Child process has not ended yet, terminating now
2015-01-18 07:01:08,957 INFO Main context doing other things now
2015-01-18 07:01:13,962 INFO Main context ended

Note that this approach above will always wait 3 seconds even if the subprocess completes sooner than that. You could convert the above into something like a loop that continually polls the child process if you want different behavior - you'll just need to keep track of how much time has elapsed.

I tried to use your code to satisfy all my constraints but I could not. Problem was that I had to put proc=subprocess.Popen in a thread but I could not access it from outside( for checking its status) since I am new to python. I wrote a code which is working. Can you please refer to it and tell whether there is any issue in using thread.exit() or any other issue in it? — user8109, Jan 21 '15 at 08:16

user8109 · Answer 4 · 2015-01-21T08:19:10.503

-1

#!/usr/bin/python

import thread
import threading
import time
import subprocess
import os 

ret=-1

def b(arg):
    global ret
    ret=subprocess.call(arg,shell=True);

thread.start_new_thread(b,("echo abcd",))
start = time.time()


while (not (ret == 0)) and ((time.time() - start)<=3):
    pass

if (not (ret == 0)) :
    print "failed"
    elapsed = (time.time() - start)
    print elapsed
    thread.exit()

elif (ret == 0):#ran before 3 sec
    print "successful"
    elapsed = (time.time() - start)
    print elapsed

I have written the above code which is working and satisfying all my contstraints. The link https://docs.python.org/2/library/thread.html says:

thread.exit() Raise the SystemExit exception. When not caught, this will cause the thread to exit silently.

So I suppose there should be no problem of orphan processes, blocked resources, etc. Please suggest.

edited Jan 21 '15 at 08:19

answered Jan 21 '15 at 08:11

user8109

189
1
12

do not use `thread` module directly, use `threading` module instead. – jfs Jan 22 '15 at 04:19
I used threading module before but I do not know how to kill thread efficiently using this module. Can u please suggest me some way to kill thread1 in the code I have posted right now? – user8109 Jan 22 '15 at 06:58
you don't kill threads. If you want to use threads; use `threading.Timer`-based solution -- *similar* to the solution from [the link that I've provided several times already](http://stackoverflow.com/a/4418891/4279). Your case is simpler, just call `process.wait()` instead of reading the output with `deque()` – jfs Jan 22 '15 at 07:09
Although both process.terminate() and process.kill() succeed in terminating the subprocess.Popen but the child process a.exe (C executable file) which is started by subprocess.Popen is not terminated. I just used a simple infinite loop as input C file. Windows Task Manager shows that a.exe consumes 50% of RAM( 1.5 GB) – user8109 Jan 22 '15 at 08:40
posted the code using process.kill and threading.Timer just now. You can try it out – user8109 Jan 22 '15 at 08:47
if you need to kill a process tree: it is a separate question (it already has an answer on Stack Overflow). Replace `process.terminate()` with that answer if you need it (it is something like `check_call("TASKKILL /F /T /PID " + str(process.pid))` on Windows). – jfs Jan 22 '15 at 08:53
Thanks for your help. Finally the code worked today evening. Posted the final code – user8109 Jan 22 '15 at 15:57

terminating a function call in python after n seconds

4 Answers4

Linked