20

Is there any obstacle that prevents weakref from doing everything that __del__ does but with much stronger guarantees (e.g., finalize guarantees that the call will be made before the interpreter exits, and the order of calls is well-defined, etc.)?

It seems that in the distant past it was thought that weakref would eventually lead to the removal of __del__ from the language.

What prevented this from happening?

There seems to be few use cases for __del__, and all the ones I'm aware of seem to work at least as well (and usually much better) with weakref callbacks or weakref.finalize.

Update:

With PEP 442 dramatically improving the behavior of __del__, and the concerns with weakref mentioned by @gz and @user2357112, I'm wondering if the language is generally moving towards making __del__ more reliable, or towards using weakref instead of __del__, or both.

Community
  • 1
  • 1
max
  • 49,282
  • 56
  • 208
  • 355
  • 1
    *"What prevented this from happening?"* that's a question that *very* few might be able to answer. I'd suggest you also post this on `python-list` to, hopefully, get the attention of some core-devs. – Dimitris Fasarakis Hilliard May 03 '17 at 11:39
  • 2
    Python's weakref support is pretty bad, so you can't take weakrefs to lots of object types you'd want to weakref. This usually doesn't matter for types you write yourself, but if your type is something like a `tuple` subclass, you can't weakref it. – user2357112 May 09 '17 at 21:39
  • 1
    Also, a weakref callback doesn't have access to the referent, so it needs more careful design. – user2357112 May 09 '17 at 21:41
  • @user2357112 For example, according to the docs, weakrefs can't be taken to generators. Does this mean that there's absolutely no way to replace generator `__del__` with weakrefs at present? Or there's some workaround (like create a separate object linked only from a generator that would serve as the trigger for `weakref.finalize`)? – max May 16 '17 at 15:06

2 Answers2

4

There's a somewhat pragmatic reason __del__ is still around. Several signficant weakref improvements, including finalize, were new in Python 3.4. So, replacing __del__ with better weakrefs missed the window for language breaking changes with py3k.

I think most uses can be replaced by the base weakref functionality, but I'm struck by this observation from Richard Oudkerk in issue 15528 where proposed and implemented finalize:

[Weakref callbacks] are low level, and working out how to use them correctly requires a bit of head scratching. One must find somewhere to store the weakref till after the referent is dead, and without accidentally keeping the referent alive. Then one must ensure that the callback frees the weakref (without leaving any remnant ref-cycles).

When it is an option, using a __del__ method is far less hassle.

Anyway, perhaps the question should be brought up again when Python 4 is being considered? ;)

Community
  • 1
  • 1
gz.
  • 6,661
  • 1
  • 23
  • 34
1

Answer to the question is really depends on the use case and would also suggest you consider that different Python interpreter implementations do not have the same GC behaviour as CPython.

PyPy in particular does not call __del__ as-soon-as an object is del-eleted or goes out of scope but 'some time later'. So code that relies on CPython's __del__ behaviour will break on PyPy, and other alternative interpreters.

What I would recommend is to make use of __enter__ and __exit__ along with the with keyword. For example

from __future__ import print_function

class MyClass(object):

    def __enter__(self):
        print("Entering")

    def __exit__(self, exc_type, exc_val, exc_tb):
        print("Exiting")

# 'with MyClass()' can be used but want to show that
# code in exit is executed regardless of object
# life time
obj = MyClass()
with obj:
    print("Stuff happening in between")

Result is:

Entering
Stuff happening in between
Exiting

The above order of statements is guaranteed and does not depend on GC behaviour.

Things that should be done as soon as the code in the with block completes can go in __exit__, such as things that would normally be put in a destructor - clearing file handles, releasing locks and so on.

A subsequent del on the object, or it going out of scope, will clear the object reference eventually, again depending on interpreter implementation, but things that should be done immediately best not rely on that behaviour.

danny
  • 5,140
  • 1
  • 19
  • 31