0

I am writing a simple threaded application and i have set the thread as daemon because I wanted my program to exit at KeyboardInterrupt. This works fine and gives expected result with python3 but python2.7 does not seem to respect the daemon flag. Below is my sample code

if __name__ == '__main__':
    try:
        threads = [Thread(target=some_func, args=(arg1,arg2)) for _ in range(10)]

        for th in threads:
            th.daemon=True
            th.start()

        for th in threads:
            th.join()
    except KeyboardInterrupt:
        sys.exit('Ctrl-c issued by user .. Exiting')

When i run this code on python3 and then hit ctrl-c after a while my program exits as expected, but when i run this with python2.7 and then hit ctrl-c it never exits and I have to kill the process from shell.

Am i missing something here ? I have also tried setting the threading.Event and then clearing the event when KeyboardInterrupt occurs but even that didn't work

With python3 If I don't join my daemon threads then they would exit as soon as the my program is done, and if I don't mark my thread as daemon and don't join then the program continues and as soon as i hit ctrl-c it exits. But none of this works with python2.7

Edit #1

I did some more digging into it and it looks like python2.7 is not able to catch KeyboardInterrupt. This is even more weird because KeyboardInterrupt works fine it is a non-threaded program but with threads the KeyboardInterrupt is not being caught even though the threads are marked as daemon.

All of this happens only on python2.7 and python3 works just fine.

Rohit
  • 3,659
  • 3
  • 35
  • 57

2 Answers2

1

The way Thread.join works in older versions of Python can silently swallow signals until the join completes. This includes SIGINT aka KeyboardInterrupt.

As a workaround, you can try short sleep or join. If you expect your threads to die by themselves, make sure to clean up:

    while threads:
        threads[-1].join(1)  # allow KeyboardInterrup once per second
        if not threads[-1].is_alive():  # reap dead Threads
           threads.pop()

My first thought was to use thread.join, but that seems to block the main process (ignoring KeyboardInterrupt) until the thread is finished.1

Multithreaded Python programs often ignore the SIGINT generated by a Keyboard Interrupt, especially if the thread that gets the signal is waiting or sleeping. 2

I'm using Python 2.7, and my program ignores SIGTERM signals when using ThreadPoolExecutor.

After some debugging I found that my main thread is blocked on Thread.join(), and it holds the interruption until the call returns. 3

MisterMiyagi
  • 44,374
  • 10
  • 104
  • 119
  • So this behavior is true for daemon threads as well ? Because it looks like this is happening for daemon threads in my case... and this seems to defeat the purpose of daemon threads then... – Rohit Dec 01 '18 at 04:02
  • This is a sideeffect of join, not the Thread itself. It does not matter whether they are daemon or not. While you can use the workaround mentioned in the answer, be aware that Python 2.7 is practically ancient. The proper fix is to use a recent version, i.e. Python 3.X. – MisterMiyagi Dec 01 '18 at 14:48
  • ofcourse i totally agree with you.. i was just testing some code on python2 and just observer this. – Rohit Dec 03 '18 at 06:10
1

Thanks to @MisterMiyagi for the suggestions and i was able to test some of the solutions mentioned in the https://stackoverflow.com/a/3788243/5349916.

Please note below solutions apply only to daemon threads because a non-daemon will always block the main program until it is done completing its work if you put the join on it. If you don't put a join on non-daemon threads then they will continue running in the background if your main has exited and this is expected behavior.

The issue I was having was for specifically for daemon threads. So here are the solutions.

# 1: Do not put a join on the thread as it will cause the thread to block the main process. The works because daemon threads exits as soon as the main program exits, so when you are not putting a join then you are not waiting in the main program and this time.sleep(1) inside while is just to make sure daemon thread keeps on executing and as soon as i hit ctrl-c, main will exit and so does daemon thread with it.

if __name__ == "__main__":
    try:
        threads = [threading.Thread(name='daemon', target=daemon)
                   for _ in range(8)]

        for th in threads:
            th.daemon = True
            th.start()

    while True:
        time.sleep(1)
    # signal.pause()    # This worked fine for me on Linux, on Windows it doesn't work, not sure about Mac. I am not recommending this because of platform dependency.

    except KeyboardInterrupt:

# 2: If you want to put a join on the thread, use timeout with it and then keep it in a while loop checking till the thread is alive. What this will do is keep on the checking every thread in threads list until its alive and then wait for 1 sec to execute it. Now since while condition is going to be True until the Thread is actually reaped after its completed; it will block the main program in small chunks of 1sec and once you hit the ctrl-c the daemon thread is not alive any more and hence main program exits. 

if __name__ == "__main__":
    try:
        threads = [threading.Thread(name='daemon', target=daemon)
                   for _ in range(8)]
        for th in threads:
            th.daemon = True
            th.start()

        for th in threads:
            while th.isAlive():
                th.join(1)

    except KeyboardInterrupt:
        print('ctrl-c')
Rohit
  • 3,659
  • 3
  • 35
  • 57
  • 1
    @MisterMiyagi i think you are you are misunderstanding. `join(1)` runs till the time thread `isAlive` returns `True`. The `join(1)` will wait for 1 second and then return and thi will happen until the thread is running. So the thread will complete its execution until i hit `ctrl-c`. – Rohit Dec 03 '18 at 06:20