38

I have searched and I'm unable to come up with any good reason to use python's __enter__ /__exit__ rather than __init__ (or __new__ ?) / __del__ .

I understand that __enter__ / __exit__ are intended for use with the with statement as context managers, and the with statement is great. But the counterpart to that is that any code in those blocks is only executed in that context. By using these instead of __init__ / __del__ I appear to be creating an implicit contract with callers that they must use with, yet there's no way to enforce such a contract, and the contract is only communicated via documentation (or reading the code). That seems like a bad idea.

I seem to get the same effect using __init__ / __del__ inside of a with block. But by using them rather than the context management methods my object is also useful in other scenarios.

So can anybody come up with a compelling reason why I would ever want to use the context management methods rather than the constructor/destructor methods?

If there's a better place to ask a question like this, please let me know, but it seems like there's not much good information about this out there.

Follow Up:

This question was based on a bad (but likely common) assumption because I always used with to instantiate a new object, in which case __init__/__del__ come very close to the same behavior as __enter__/__exit__ (except that you can't control when or if __del__ will be executed, it's up to garbage collection and if the process is terminated first it may never be called). But if you use pre-existing objects in with statements they are of course quite different.

BobDoolittle
  • 1,326
  • 1
  • 12
  • 21

3 Answers3

23

There are several differences you appear to have missed:

  • Context manager get a chance to provide a new object just for the block you are executing. Some context managers just return self there (like file objects do), but, as an example, database connection objects could return a cursor object tied to the current transaction.

  • Context managers are not just notified of the context ending, but also if the exit was caused by an exception. It can then decide on handling that event or otherwise react differently during exit. Using a database connection as an example again, based on there being an exception you could either commit or abort the transaction.

  • __del__ is only called when all references to an object are removed. This means you can't rely on it being called if you need to have multiple references to it that you may or may not control the lifetime of. A context manager exit is precisely defined however.

  • Context managers can be reused, and they can keep state. The database connection again; you create it once, then use it as a context manager again and again, and it'll keep that connection open. There is no need to create a new object each time for this.

    This is important for thread locks, for example; you have to keep state so that only one thread can hold the lock at a time. You do this by creating one lock object, then use with lock: so different threads executing that section each can be made to wait before entering that context.

The __enter__ and __exit__ methods form the context manager protocol, and you should only use these if you actually want to manage a context. The goal of context managers is to simplify common try...finally and try...except patterns, not to manage the lifetime of a single instance. See PEP 343 – The "with" Statement:

This PEP adds a new statement "with" to the Python language to make it possible to factor out standard uses of try/finally statements.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • Your first two points are really about the `with` statement, not enter and exit. I agree that `with` is terrific, but in this question I'm more concerned about the best way to write a flexible object that can be used inside `with` or not. I get the `__del__` point. – BobDoolittle Nov 10 '16 at 21:10
  • 3
    @BobDoolittle: there is little point in implementing `__enter__` and `__exit__` **without** the `with` statement. The `with` statement is the reason we have those methods in the first place. – Martijn Pieters Nov 10 '16 at 21:14
  • Of course there is. It's the difference between a server and a client. Between a caller and a service. I write an object. Somebody else may use it. They may use it in a `with` block or not, and I have no way to enforce usage. I should write my object in such a way that it will perform correctly in either case. Like `File` objects - I can use then in a `with` statement or not, and they work either way. Outside of a `with` block I should call close() however (but I suspect `__del__` does this also). – BobDoolittle Nov 10 '16 at 21:18
  • @BobDoolittle: the `__enter__` and `__exit__` methods are a *protocol*. You'd only implement those if you wanted to support using your object as a context manager. You can't use an object as a context manager otherwise. Don't implement the protocol if you don't want to be a context manager. This is the same choice as implementing the mapping or container or number protocols. You'd implement those if your use case warrants it. – Martijn Pieters Nov 10 '16 at 21:25
  • In that case there should be some language enforcement - e.g. throw an exception when instantiating such an object outside of `with`. Again I fall back to the `File` object example. It does not force you to use `with`. You can directly instantiate and use it. But in that case you should probably be more careful (e.g. call close() explicitly). To create such behavior, it must not rely on `__enter__`. – BobDoolittle Nov 10 '16 at 21:28
  • @BobDoolittle: Why would that be needed? There is an exception when you want to use an object in a `with` statement that is not a context manager. You don't **have** to create the context manager in the `with` statement. `lock = threading.Lock()` is fine, then `with lock:` lets you control locking for a context. – Martijn Pieters Nov 10 '16 at 21:30
  • 1
    @BobDoolittle: You seem to be conflating *creating* a context manager with using it to manage a context. The two are *separate steps*. It is often convenient to open a file and use it as a context manager at the same time, but you don't have to. Why would you have to? Locks can be used as context manager again and again and again. See [Other builtin or practical examples of python \`with\` statement usage?](//stackoverflow.com/a/19484315) for more examples of context managers. – Martijn Pieters Nov 10 '16 at 21:31
  • Yes you are right. I am used to instantiating a new object in a `with` statement, but that's not the only usage pattern. Makes total sense, thanks. – BobDoolittle Nov 10 '16 at 21:34
7

del x doesn’t directly call x.__del__()

You have no control over when .__del__ is called, or in fact whether it gets called at all.

Therefore, using __init__/__del__ for context management is not reliable.

wim
  • 338,267
  • 99
  • 616
  • 750
  • I understand your point about `__del__`. But not wrt `__init__`. `__init__` should be reliable inside or outside of context management. At this point, I'm thinking the best way to write a flexible object is to put your initialization code in `__init__`, rather than `__enter__`, and to make `__exit__` do the same thing as `__del__` (while protecting against duplicate execution). In fact, I suspect that's exactly what the File objects do. – BobDoolittle Nov 10 '16 at 21:11
  • 3
    @BobDoolittle: `__init__` is for initialization, so if you want to do initialization, do it there. `__enter__` is for work that should happen specifically on entering a `with` statement; for example, `__enter__` might lock a lock and `__exit__` unlock it. – user2357112 Nov 10 '16 at 21:34
3

By using these instead of __init__ / __del__ I appear to be creating an implicit contract with callers that they must use with, yet there's no way to enforce such a contract

You have a contract either way. If users use your object without realizing it requires cleanup after use, they'll screw things up no matter how you implement the cleanup. They might keep a reference to your object forever, for example, preventing __del__ from running.

If you have an object that requires special cleanup, you need to make this requirement explicit. You need to give users with functionality and an explicit close or similar method, to let users control when the cleanup occurs. You can't hide the cleanup requirement inside a __del__ method. You might want to implement __del__ too, as a safety measure, but you can't use __del__ in place of with or an explicit close.

With that said, Python makes no promises that __del__ will run, ever. The standard implementation will run __del__ when an object's refcount drops to 0, but that might not happen if a reference survives to the end of the script, or if the object is in a reference cycle. Other implementations don't use refcounting, making __del__ even less predictable.

user2357112
  • 260,549
  • 28
  • 431
  • 505