Multiprocessing Process terminate fails on Linux

Question

I just noticed the problem with process terminate (from multiprocessing library) method on Linux. I have application working with multiprocessing library but... when I call terminate function on Windows everything works great, on the other hand Linux fails with this solution. As a replacement of process killing I was forced to use

os.system('kill -9 {}'.format(pid))

I know this isn't too smart, but it works. So I just wondering why this code works on Windows, but on Linux fails.

Example:

from multiprocessing import Process
import os

process=Process(target=foo,args=('bar',))
pid=process.pid
process.terminate() # works on Windows only

...

os.sytem('kill -9 {}'.format(pid)) # my replacements on Linux

My configuration: python 3.2.0 build 88445; Linux-2.6.32-Debian-6.0.4

This is a sample from my code. I hope it will be sufficient.

def start_test(timestamp,current_test_suite,user_ip):
    global_test_table[timestamp] = current_test_suite
    setattr(global_test_table[timestamp], "user_ip", user_ip)
    test_cases = global_test_table[timestamp].test_cases_table

    test_cases = test_cases*int(global_test_table[timestamp].count + 1)
    global_test_table[timestamp].test_cases_table = test_cases
    print(test_cases)
    print(global_test_table[timestamp].test_cases_table)

    case_num = len(test_cases)
    Report.basecounter = Report.casecounter = case_num

    setattr(global_test_table[timestamp], "case_num", case_num)
    setattr(global_test_table[timestamp], "user_current_test", 0)

    try:
        dbobj=MySQLdb.connect(*dbconnector)
        dbcursor=dbobj.cursor()

        dbcursor.execute(sqlquery_insert_progress.format(progress_timestamp = str(timestamp), user_current_test = global_test_table[timestamp].user_current_tes$
    except :...

    for i in range(case_num):
        user_row = global_test_table[timestamp]
        current_test_from_tests_table = user_row.test_cases_table[i]
        unittest.TextTestRunner(verbosity=2).run(suite(CommonGUI.get_address(CommonGUI,current_test_from_tests_table[1], current_test_from_tests_table[2], user$
        global_test_table[timestamp].user_current_test = i + 1
        try:
            dbobj=MySQLdb.connect(*dbconnector)
            dbcursor=dbobj.cursor()

            dbcursor.execute(sqlquery_update_progress.format(progress_timestamp = str(timestamp), user_current_test = global_test_table[timestamp].user_current$
        except :...

@cherrypy.expose()
def start_test_page(self, **test_suite):
    timestamp = str(time.time())
    user_ip = cherrypy.request.remote.ip
    if on_server():
        sys.stdout=sys.stderr=open("/var/log/cherrypy/test_gui/{file}.log".format(file=timestamp),"a")
    current_test_suite = self.parse_result(**test_suite)
    #global_test_table[timestamp] = current_test_suite
    #setattr(global_test_table[timestamp], "user_ip", user_ip)
    user_test_process = Process(target=start_test, args=(timestamp,current_test_suite,user_ip))
    users_process_table[timestamp] = user_test_process
    user_test_process.start()
    return '''{"testsuite_id" : "''' + str(timestamp) + '''"}'''

@cherrypy.expose()
def stop_test(self, timestamp):
    if timestamp in users_process_table:
        if on_server():
            user_process_pid = users_process_table[timestamp].pid
            os.system("kill -9 " + str(user_process_pid))
        else:
            users_process_table[timestamp].terminate()
        del users_process_table[timestamp]
    else:
        return "No process exists"

Can you post more of your code? It'd be helpful to know what foo is doing with bar, and from that we would probably gain a better idea of why Linux isn't killing it, but Windows is. — parselmouth, May 10 '12 at 07:56

codeape · Accepted Answer · 2012-05-10T11:49:11.667

From the docs:

terminate()

Terminate the process. On Unix this is done using the SIGTERM signal; on Windows TerminateProcess() is used. Note that exit handlers and finally clauses, etc., will not be executed.

Note that descendant processes of the process will not be terminated – they will simply become orphaned.

So it looks like you have to make sure that your process handles the SIGTERM signal correctly.

Use signal.signal to set a signal handler.

To set a SIGTERM signal handler that simply exist the process, use:

import signal
import sys
signal.signal(signal.SIGTERM, lambda signum, stack_frame: sys.exit(1))

EDIT

A Python process normally terminates on SIGTERM, I don't know why your multiprocessing process doesn't terminate on SIGTERM.

score 0 · Answer 2 · edited May 23 '17 at 11:54

0

Not exactly a direct answer to your question, but since you are dealing with the threads this could be helpful as well for debugging those threads: https://stackoverflow.com/a/10165776/1019572 I recently found a bug in cherrypy using this code.

edited May 23 '17 at 11:54

Community

1
1

answered May 11 '12 at 05:51

Stefan

326
2
4

Multiprocessing Process terminate fails on Linux

2 Answers2

Linked