Python iter and for loops

Question

As I understand it, I can use the for loop construction on an object with a __iter__ method that returns an iterator. I have an object for which I implement the following __getattribute__ method:

def __getattribute__(self,name):
    if name in ["read","readlines","readline","seek","__iter__","closed","fileno","flush","mode","tell","truncate","write","writelines","xreadlines"]:
        return getattr(self.file,name)
    return object.__getattribute__(self,name)

I have an object of this class, a for which the following happens:

>>> hasattr(a,"__iter__")
True
>>> for l in a: print l
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: 'TmpFile' object is not iterable
>>> for l in a.file: print l
...
>>>

So python sees that a has an __iter__ method, but doesn't think it is iterable. What have I done wrong? This is with python 2.6.4.

score 12 · Accepted Answer · answered Mar 14 '11 at 16:08

12

There's a subtle implementation detail getting in your way: __iter__ isn't actually an instance method, but a class method. That is, obj.__class__.__iter__(obj) is called, rather than obj.__iter__().

This is due to slots optimizations under the hood, allowing the Python runtime to set up iterators faster. This is needed since it's very important that iterators be as fast as possible.

It's not possible to define __getattribute__ for the underlying class type, so it's not possible to return this method dynamically. This applies to most __metamethods__; you'll need to write an actual wrapper.

answered Mar 14 '11 at 16:08

Glenn Maynard

55,829
10
121
131

1

+1 So that's why `iter` doesn't call `__getattribute__`... as they say, you learn something new on SO every day :) – Mar 14 '11 at 16:13
Note that you can see a side-effect of this (at least in CPython): if you define `__getattribute__`, you'll notice it gets called for `__class__` when you use `iter`, as the runtime looks up the class of the object to call `__iter__`. (This should probably be considered an implementation detail.) – Glenn Maynard Mar 14 '11 at 16:58

score 4 · Answer 2 · answered Mar 14 '11 at 16:12

Some of the special methods are optimised when a class is created and cannot be added later or overridden by assignment. See the documentation for __getattribute__ which says:

This method may still be bypassed when looking up special methods as the result of implicit invocation via language syntax or built-in functions.

What you need to do in this case is provide a direct implementation of __iter__ that forwards the call:

def __iter__(self):
    return self.file.__iter__()

Python iter and for loops

2 Answers2

Linked

Python __iter__ and for loops

2 Answers2

Linked

Python iter and for loops