When calling a python function from another function, how do I let the calling function know of an error?

Question

I am trying to figure out in Python (using 3.8) how to pass an error from a function to the function that called it. I have a try /except which is catching the exception and writing to a log file, but the parent function seems to not be able to get the value of a variable to indicate there was an error. How can I pass this value to the parent function? Should I be using a different type of variable?

I have the following setup: a

1. a variable (BOOL) named any_errors, starts set at False
2. main function (foo.py)
3. a class (referred to below as sf_dw) with functions within a different file(utils/snowflake.py).

The CONTROL_TABLE_NAME table has two columns, target_object and query (a full SQL statement to be executed). megre_to_dw just retrieves these values, as is not shown.

In this case, my main function makes a call from foo.py as such:

    try:
        control_table='CONTROL_TABLE_NAME'

        dict_of_merge_queries = sf_dw.merge_to_dw(control_table, 'STREAM')

        executor = concurrent.futures.ThreadPoolExecutor(max_workers=8)
        futures = [executor.submit(sf_dw.merge_to_dw_threads, target_object, query, control_table, file) for target_object, query in dict_of_merge_queries.items()]
        concurrent.futures.wait(futures)

    except Exception as e:
        file.write("Error Written By Calling function: " + str(e) + "\n")
        any_errors= True
        file.write("Setting any_errors True 4 \n")

The called function is:

def merge_to_dw_threads(self, target_object, query, control_table, file): 

        try:
           self.sf_cur.execute(query)

        except Exception as e:
            file.write("Error Written By Inner function:" + str(e) + "\n")
            any_errors= True
            file.write("Setting any_errors True 5 \n")
            return any_errors

Later I write this to the file:

file.write(" Any_Errors status is " + str(any_errors) + "\n")

I then use a statement at the end of my main function to produce an exit code based on the value of the variable "any_errors".

The problem I have is that the fact that there was an error in the 'merge_to_dw_threads' function, though it is written to the file, never makes it up to the calling function and the program exists reporting no errors.

The Error file reads like this:

Error Written By Inner function:SQL compilation error:
Schema 'XXXX.XXX' does not exist or not authorized.
Setting any_errors True 5 
 Any_Errors status is False

Answer 1

That is what the try catch is intended to do. You can do a few different things, depending what you want to achieve.

1. You can re-throw the exception Manually raising (throwing) an exception in Python

2. You can create an instance variable in the constructor to check and track whether the crash has occurred

3. You can return a value you know means an error occurred, and check it conditionally (you'd have to assign it before using it in the other function so you can check it)

I would recommend 1, since it seems like this failing should throw an exception and stop execution. You could also do various other things, like exit the program.