Pythonic way to close connection-like objects in __del__

Question

I'm working on a connection-like object which implements a context manager. Writing something like this is strongly encouraged:

with MyConnection() as con:
    # do stuff

Of course one can do this as well:

con = MyConnection()
# do stuff
con.close()

But failing to close the connection is rather problematic. So closing in the __del__() seems like a good idea:

def __del__(self):
    self.close()

This looks quite nice, but sometimes leads to errors:

Exception ignored in: [...]
Traceback (most recent call last):
  File "...", line xxx, in __del__()
TypeError: 'NoneType' object is not callable

It appears as if sometimes the close method is already destroyed, when __del__() is called.

So I'm looking for a nice way to encourage python to close the connection properly on destruction. If possible I would like to avoid code duplication in close() and __del__()

Answer 1

If you really want to prevent the user from not closing the connection, you could just init it only in __enter__ or you may add a flag spotting the fact it has not been initialized by a context manager. For instance, something like

class MyConnection(object):

    safely_initialized = False

    def __enter__(self):
        # Init your connection
        self.safely_initialized = True
        return self

    def do_something(self):
        if not self.safely_initialized:
            raise Exception('You must initialize the connection with a context manager!')
        # Do something

    def __exit__(self, type, value, traceback):
        # Close your connection

That way the connection won't be initialized unless within a context manager.

Answer 2

There is no guarantee when __del__ will actually run. Since you are using a with statement, use the __exit__ method instead. __exit__ will be called as soon as the with statement is finished, no matter how the statement completes (normally, with an exception, etc).

Answer 3

You could try calling close in __del__, and ignore any exceptions:

del __del__(self):
    try:
        self.close()
    except TypeError:
        pass

Answer 4

It is true you cannot force your users to use good programming techniques, and if they refuse to do so you cannot be responsible for them.

There is no guarantee of when __del__ will be called -- in some Pythons it is immediate, in others it may not happen until interpreter shutdown. So, while not a very good option, using atexit may be viable -- just be sure that the function you register is smart enough to check if the resource has already been closed/destroyed.

Answer 5

weakref.finalize() lets you perform an action when an object is garbage collected, or the program exits. It's available from Python 3.4 onwards.

When you create your object you can make a call to finalize(), providing it a callback that cleans up the resources your object holds.

The Python docs provide several examples of its use:

>>> import weakref
>>> class Object:
...     pass
...
>>> kenny = Object()
>>> weakref.finalize(kenny, print, "You killed Kenny!")  
<finalize object at ...; for 'Object' at ...>
>>> del kenny
You killed Kenny!

And this example of a class representing a temporary directory whose contents are deleted when:

• its remove() method is called
• it gets garbage collected
• program exits

Whichever happens first.

class TempDir:
    def __init__(self):
        self.name = tempfile.mkdtemp()
        self._finalizer = weakref.finalize(self, shutil.rmtree, self.name)

    def remove(self):
        self._finalizer()

    @property
    def removed(self):
        return not self._finalizer.alive

Answer 6

You may define a close() method and call it on is_open condition both in __exit__ and __del__ methods as follows:

class MyContext:
    def __init__(self, *args):
        self.is_open = False
        self.args = args
        self.open(*self.args)
    def __enter__(self):
        return self
    def __del__(self):
        self.close()
    def __exit__(self, *args):
        self.close()
    def open(self, *args):
        if not self.is_open:
            self.is_open = True
            print("opening: ", args)
        return self
    def close(self):
        if self.is_open:
            self.is_open = False
            print("closing: ", self.args)

---

Here is a usage example WITHOUT a context manager:

def init_way():
    c = MyContext("path", "openparam")

init_way()

Possible output:

opening:  ('path', 'openparam')
closing:  ('path', 'openparam')

---

And another example: using the same class as a context manager this time:

def context_way():
    with MyContext("path", "openparam") as c:
        print("in with ...")

context_way()

Possible output:

opening:  ('path', 'openparam')
in with ...
closing:  ('path', 'openparam')