9

I'm trying to perform checks on the first and last elements of an interator. It has several thousand entries, so I need an expeditious method of checking. If found this post, that put me onto this strategy.

first = True
for value in iterator:
   if first:
      do_stuff_to_first_iter
      first = False
   else:
      pass
do_stuff_to_last_iter

Anyone have any opinions on a quicker method of accomplishing this? Thanks a lot!

Community
  • 1
  • 1
Rich
  • 93
  • 1
  • 1
  • 3

4 Answers4

30

Get the first value with the next() function:

first = last = next(iterable, defaultvalue)
for last in iterable:
    pass

This assumes the iterable is finite.

For an empty iterable, first and last are set to defaultvalue. For an iterable with just one element, first and last will both refer to that one element. For any other finite iterable, first will have the first element, last the very last.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • Hello Martijn, thanks for the answer. This technique is very pythonic and (now that I've worked out all the bugs) seems to work very well. It still takes about 4-minute to loop through my iterator, but it's better than what I had before. Thanks again. – Rich Nov 20 '13 at 20:13
  • OK, so I timed this against Paul's `deque` method. This method took my process 264 seconds. The `deque` method took 275 seconds. Within the noise so to speak. But I'm going to use this guy! Thanks. – Rich Nov 20 '13 at 21:26
3

Based on my answer to the linked question:

Probably worth using __reversed__ if it is available. If you are providing the iterator, and there is a sensible way to implement __reversed__ (ie without iterating from end to end) you should do it

first = last = next(my_iter)
if hasattr(my_iter,'__reversed__'):
    last = next(reversed(my_iter))
else:
    for last in my_iter:
        pass
John La Rooy
  • 295,403
  • 53
  • 369
  • 502
  • 1
    `__reversed__` is commonly only found on objects that are not themselves iterators. E.g. `list` provides it, but not `iter(list)`. – Martijn Pieters Nov 20 '13 at 19:44
  • 1
    In the standard library, only `list()`, `range()`, `collections.deque()`, and `collections.OrderedDict()` provide a `__reversed__`, and none are themselves iterators. That means you **cannot** call `next()` on them. – Martijn Pieters Nov 20 '13 at 19:49
  • I like the option of reversing the iterator and avoiding unnecessary loops. Unfortunately, my iterator does not have a `__reversed__` attribute and I'm left entering the loop anyway. Thanks for the suggestion! – Rich Nov 20 '13 at 20:07
1

You can use a deque with a maxlen of 1 to quickly get the last element of a finite iterator:

>>> from collections import deque
>>> last_getter = deque(maxlen=1)
>>> seq = range(10000)
>>> iseq = iter(seq)
>>>
>>> first = last = next(iseq, None)
>>> last_getter.extend(iseq)
>>> if last_getter: last = last_getter[0]
...
>>> print (first, last)
0 9999
Cristian Ciupitu
  • 20,270
  • 7
  • 50
  • 76
PaulMcG
  • 62,419
  • 16
  • 94
  • 130
  • 1
    So I tried this method and timed it for my process. Took 275 seconds. The "loop n' pass" technique took 264 seconds. Either way, both are comparable. Thanks for suggesting! This is an interesting technique I didn't know existed. – Rich Nov 20 '13 at 21:27
1

According to my tests, islice is up to 3 times faster than for: pass or deque. This would require you to know how many items there will be, though.

last = next(islice(iterable, length - 1, length))

Or if you don't know the full length, but do know that it must be at least n, you could still "skip" to n as a shortcut:

rest = islice(iterable, n, None)
last = next(rest)
for last in rest:
    pass
nmclean
  • 7,564
  • 2
  • 28
  • 37