Intercepting getitem calls on an object attribute

Question

Question: How can I Intercept __getitem__ calls on an object attribute?

Explanation:

So, the scenario is the following. I have an object that stores a dict-like object as an attribute. Every time the __getitem__ method of this attribute gets called, I want to intercept that call and do some special processing on the fetched item depending on the key. What I want would look something like this:

class Test:

    def __init__(self):
        self._d = {'a': 1, 'b': 2}

    @property
    def d(self, key):
        val = self._d[key]
        if key == 'a':
            val += 2
        return val
t = Test()
assert(t.d['a'] == 3) # Should not throw AssertionError

The problem is that the @property method doesn't actually have access to the key in the __getitem__ call, so I can't check for it at all to do my special postprocessing step.

Important Note: I can't just subclass a MutableMapping, override the __getitem__ method of my subclass to do this special processing, and store an instance of the subclass in self._d. In my actual code self._d is already a subclass of MutableMapping and other clients of this subclass need access to the unmodified data.

Thanks for any and all help!

The dupe explains how to create a *descriptor*, which `@property` is a specific case of, allowing you to hook into the `__get__` process however you want. — jonrsharpe, Jul 03 '18 at 17:01
@jonrsharpe: That's only tangential to the goal here though. You don't really need to write your own descriptor here; there are other options. Worthy of linkage, but not a duplicate. — ShadowRanger, Jul 03 '18 at 17:03

score 4 · Accepted Answer · answered Jul 03 '18 at 17:10

One solution would be a Mapping that proxies the underlying mapping. The d property would wrap the underlying self._d mapping in the proxy wrapper and return it, and use of that proxy would exhibit the necessary behaviors. Example:

from collections.abc import Mapping

class DProxy(Mapping):
    __slots__ = ('proxymap',)
    def __init__(self, proxymap):
        self.proxymap = proxymap
    def __getitem__(self, key):
        val = self.proxymap[key]
        if key == 'a':
            val += 2
        return val
    def __iter__(self):
        return iter(self.proxymap)
    def __len__(self):
        return len(self.proxymap)

Once you've made that, your original class can be:

class Test:
    def __init__(self):
        self._d = {'a': 1, 'b': 2}

    @property
    def d(self):
        return DProxy(self._d)

Users would then access instances of Test with test.d[somekey]; test.d would return the proxy, which would then modify the result of __getitem__ as needed for somekey. They could even store off references with locald = test.d and then use locald while preserving the necessary proxy behaviors. You can make it a MutableMapping if needed, but a plain Mapping-based proxy avoids complexity when the goal is reading the values, never modifying them through the proxy.

Yes, this makes a new DProxy instance on each access to d; you could cache it if you like, but given how simple the DProxy class's __init__ is, the cost is only meaningful if qualified access via the d attribute is performed frequently on the hottest of code paths.

This is very similar to the solution I settled upon whilst my question was closed as a duplicate, but I have a question. Why return a new DProxy every time? It seems like I might as well just create a DProxy and store that as self.d and drop all the @property business entirely. — Mr. Frobenius, Jul 03 '18 at 23:18
@Mr.Frobenius: That's a reasonable solution, though it makes it possible for someone to *assign* to `d` (so you might want an `@property` regardless to impose the read-only behavior in the API). It also means eagerly creating (or lazily creating via `property` or `__getattr__`), and preserving, the proxy wrapper, which, if it's only used occasionally (sometimes not at all) might not be ideal. If that's not a problem though, sure, go nuts! :-) — ShadowRanger, Jul 03 '18 at 23:24

Silvio Mayolo · Answer 2 · 2018-07-03T17:23:22.450

Here's a fairly similar approach to ShadowRanger's. It's a bit shorter, as it inherits from dict directly, so there's less explicit delegation to define.

class DictProxy(dict):
    def __getitem__(self, item):
        val = super().__getitem__(item)
        if item == 'a':
            val += 2
        return val

class Test:

    def __init__(self):
        self._d = {'a': 1, 'b': 2}

    @property
    def d(self):
        return DictProxy(self._d)

t = Test()
assert(t.d['a'] == 3) # Does not throw AssertionError anymore :)

~~In terms of behavior, it really comes down to taste. There's nothing wrong with either approach.~~

EDIT: Thanks to ShadowRanger for pointing out that this solution actually copies the dictionary every time. Therefore, it's probably better to use his explicit delegation solution, which uses the same internal dictionary representation. It'll be more efficient that way, and if you ever want to change your proxy in the future so that it actually affects the original data structure, his approach will make it a lot easier to make those future changes.

There is a meaningful difference in behavior to mention: This approach is snapshot based; it creates a brand new instance of a `dict` subclass on each access to `d`, shallow copying the current values of `self._d`. That increases the cost of each separate access to `t.d` (especially if `t._d` is a large `dict`), and if `t.d` is cached, modifications to `self._d` are not reflected in the cached copy. Whether that's appropriate is dependent on use case. — ShadowRanger, Jul 03 '18 at 17:19
I honestly had no idea there was a shallow copy happening in there. I've used this trick myself several times without realizing that. I'll leave my answer here for completeness, but I concede that your way is the superior way to approach this. — Silvio Mayolo, Jul 03 '18 at 17:21

score 0 · Answer 3 · answered Jul 03 '18 at 17:50

0

No shallow copying, shortest, and with modification possibilities:

from collections import UserDict

class DictProxy(UserDict):
    def __init__(self, d):
        self.data = d

    def __getitem__(self, item):
        val = super().__getitem__(item)
        if item == 'a':
            val += 2
        return val

answered Jul 03 '18 at 17:50

Bharel

23,672
5
40
80

@SilvioMayolo it doesn't, you have a bug in your code. Use my class. – Bharel Jul 03 '18 at 18:14
You didn't use it. – Bharel Jul 03 '18 at 18:19
*sigh* I apologize for my rather foolish typo there. Carry on :) – Silvio Mayolo Jul 03 '18 at 18:19
@SilvioMayolo: It should work, though it's a little worrisome in that it violates one of the documented rules of `UserDict` (`UserDict` itself is documented to provide a *plain* `dict` in `self.data`; only by overriding `__init__` and failing to invoke the super class version is that avoided). I'm pretty sure it should work under the current implementation (I checked the code for any obvious misbehaviors this might cause and didn't see any), but "works on given implementation" always gives me the heebie-jeebies. – ShadowRanger Jul 03 '18 at 18:21
@ShadowRanger You may create a function which creates an empty DictProxy and replacing `.data` underneath. Doesn't matter much. – Bharel Jul 03 '18 at 18:38

Intercepting getitem calls on an object attribute

3 Answers3

Linked

Intercepting __getitem__ calls on an object attribute

3 Answers3

Linked

Intercepting getitem calls on an object attribute