how to handle "Too many values to unpack"

Question

I have checked some idea and the reason, is which investigated below for this problem... "Too many values to unpack" Exception (Stefano Borini's explanation)

But here I am iterating through a list as a comprehension list and move the result to a list...! So the number of the inputs reads the number of the output variable, i.e. tempList...

Then, what is wrong with the process?!

def DoProcess(self, myList):
    tempList = []
    tempList = [[x,y,False] for [x,y] in myList]
    return tempList

Edit 1: myList is a list of lists, just like [[x1, y1], [x2, y2], [x3, y3], [x4 y4]].

class Agent(object):
    def __init__(self, point = None):
        self.locationX = point.x
        self.locationY = point.y

    def __iter__(self):
        return self

    def __next__(self):
        return [self.locationX, self.locationY]

    def __getItem__(self):
        return [self.locationX, self.locationY]

    def GenerateAgents(self, numberOfAgents):
        agentList = []
        while len(agentList) < numberOfAgents:

            point = Point.Point()
            point.x = random.randint(0, 99)
            point.y = random.randint(0, 99)

            agent = Agent(point)
            agentList.append(agent)
        return agentList

    def DoProcess(self, myList):
        tempList = []
        tempList = [[x[0],x[1],False] for x in myList]
        return myList

And each Point has two attribute as locationX and locationY...

Does myList contain a list of 2 element tuples? If they has more than 2 elements each then you'll run into trouble. — SuperBiasedMan, May 11 '15 at 14:24
@Ordenador: then your input is not actually conforming to your specification. Can you share what `myList` really is? — Martijn Pieters, May 11 '15 at 14:28
@Ordenador You should print exactly what's in the list, apparently you have a list of atomic objects. Or you have a list of mixed objects. — Loïc Faure-Lacroix, May 11 '15 at 14:32
@MartijnPieters: here is the concrete view of `myList`, as I edit the question... — User, May 11 '15 at 14:36

Martijn Pieters · Accepted Answer · 2015-05-11T15:16:36.840

5

Your implementation of Agent is severely flawed; you created an infinite generator:

def __iter__(self):
    return self

def __next__(self):
    return [self.locationX, self.locationY]

This will forever yield lists with two values. Trying to use this object in a tuple assignment will yield at least 3 such values (2 for the x and y targets, plus one more for Python to know there were more values to unpack than requested). What Python does is call __next__ each time it needs another value in the sequence, and your code just returns [x, y] each time. For ever and ever until eternity.

The __iter__ method should return an actual iteration over the two values instead:

def __iter__(self):
    for value in (self.locationX, self.locationY):
        yield value

or even just

def __iter__(self):
    yield self.locationX
    yield self.locationY

dropping the __next__ altogether. The above generator will then yield two values then raise StopIteration properly, and work with tuple assignment.

The __getitem__ method is spelled all lowercase and takes an index argument:

def __getitem__(self, index):
    return (self.locationX, self.locationY)[index]

Now 0 maps to locationX and 1 to locationY.

Rewriting your code with those changes:

class Agent(object):
    def __init__(self, point):
        self.locationX = point.x
        self.locationY = point.y

    def __iter__(self):
        yield self.locationX
        yield self.locationY

    def __getitem__(self, index):
        return (self.locationX, self.locationY)[index]

    def GenerateAgents(self, numberOfAgents):
        agentList = []
        for _ in range(numberOfAgents):
            point = Point.Point()
            point.x = random.randint(0, 99)
            point.y = random.randint(0, 99)

            agent = Agent(point)
            agentList.append(agent)
        return agentList

    def DoProcess(self, myList):
        return [[x, y, False] for x, y in myList]

edited May 11 '15 at 15:16

answered May 11 '15 at 14:25

Martijn Pieters

1,048,767
296
4,058
3,343

For "2 or more", can't he unpack to `x,y,*_`? – Stefan Pochmann May 11 '15 at 14:34
1

I deeply appreciate for your explanation... But even I have implemented `__getItem__()` method, I cant iterate by `[[elem[0], elem[1], False] for elem in myList]` because of `Object does not support indexing` – User May 11 '15 at 14:34
@Ordenador: you misspelled the method name there; The `i` is *lowercase*. – Martijn Pieters May 11 '15 at 14:37
@Ordenador: your implementation of `__getitem__` is very much flawed; you don't return a list, you return the item at the given index, but you did not specify the argument to *receive* the given index. – Martijn Pieters May 11 '15 at 14:42
@MartijnPieters: Would you please let me know how to rephrase `__getitem__()`?... – User May 11 '15 at 14:44
@Ordenador: rewritten my answer entirely. – Martijn Pieters May 11 '15 at 14:51
@MartijnPieters wouldn't it be better to just drop the `iterator` and `__getitem__`? – Loïc Faure-Lacroix May 11 '15 at 18:31
@LoïcFaure-Lacroix: then you'd need to implement `__len__` too. – Martijn Pieters May 11 '15 at 18:36
@MartijnPieters I mean using the object attributes instead of unpacking things. – Loïc Faure-Lacroix May 12 '15 at 00:20
@LoïcFaure-Lacroix: there are many better patterns to explore here yes; I was focusing on the unpacking however and why it doesn't work. – Martijn Pieters May 12 '15 at 06:59

score 1 · Answer 2 · answered May 11 '15 at 14:26

1

Your list needs to contain nested iterables of length two to which x and y are unpacked.

answered May 11 '15 at 14:26

Malik Brahimi

16,341
7
39
70

Loïc Faure-Lacroix · Answer 3 · 2015-05-11T18:27:27.367

The proper way to make your getitem method is to write it as such:

def __getitem__(self, index):
    if index == 0:
       return self.locationX
    if index == 1:
       return self.locationY
    raise IndexError()

Note that it has an index passed as argument and it is written __getitem__ and not __getItem__. Without the index error, it seems that python tries to unpack as many values as possible until the getitem raise an index error.

Not that you can simplify your code and add a clause for the index 2 and return False.

Honestly, I don't see the point to override getitem here. It will be easier to understand if you write.

tempList = [[x.locationX,x.locationY,False] for x in myList]

Also there is no need to write this:

tempList = []
tempList = [...]

Creating an empty list to replace it by a new list is pointless.

Here's a reworked sample of code. Note that I changed the method Generate and DoProcess as staticmethod. They can be made static as they do not really require any instance to work. You can call them directly using the Agent class. I removed the iterator, getitem as they aren't really necessary here. If they are used anywhere else then it might create troubles.

The thing is that in this case, it seems strange to unpack values from an Agent. If I would ever see such code... I wouldn't understand the needs for an iterator or __getitem__. It's not obvious that agent[0] is its X location and agent[1], its Y location. Python has named attributes for a reason. If you don't use them, then you could simply store your agents in a list instead of a class. Well that's exactly what the DoProcess method do.

class Agent(object):

    def __init__(self, point):
        self.locationX = point.x
        self.locationY = point.y

    @staticmethod
    def GenerateAgents(numberOfAgents):
        agentList = []

        for i in range(numberOfAgents):
            point = Point.Point()
            point.x = random.randint(0, 99)
            point.y = random.randint(0, 99)

            agent = Agent(point)
            agentList.append(agent)

        return agentList

    @staticmethod
    def DoProcess(myList):
        return [
            [obj.locationX, obj.locationY, False]
            for obj in myList
        ]

@Ordenador thanks, I rewrote your class to simplify it. – Loïc Faure-Lacroix May 11 '15 at 18:34 — Loïc Faure-Lacroix, May 11 '15 at 18:34

score 0 · Answer 4 · answered May 11 '15 at 14:25

0

Maybe something like this:

def DoProcess(self, myList):
    tempList = [[x[0],x[1],False] for x in myList]
    return tempList

answered May 11 '15 at 14:25

user2707389

817
6
12

Thank you... but even with implementation of `__getItem__()` and `__iter__()` functions there is an exception as `Object does not support indexing`... – User May 11 '15 at 14:29
@Ordenador this means that you have objects that aren't iterable in your list. Which means that can't be unpacked to 2 items – Loïc Faure-Lacroix May 11 '15 at 14:30
@LoïcFaure-Lacroix: But I should be able to... I have implemented `__getItem__()` method... Am I right?!... – User May 11 '15 at 14:32

tennabey · Answer 5 · 2015-05-11T14:39:30.863

You could just do this

def DoProcess(self, myList):
    return [sublist + [False] for sublist in myList]

There are several options as to why this isn't working:

Somewhere in myList you have sublists which have fewer than two elements (or more generally, sublists whose length isn't 2). You can find them with
```
print [sublist for sublist in myList if len(sublist) != 2]
```

There are elements in myList which aren't lists. You can find them with

print [element for element in myList if not isinstance(element, list)]

Combine them together with

print [element for element in myList if not isinstance(element, list) or len(element) != 2]

That is not supposed to work... because the `[False]` type is not compatible with the `sublist` type... — User, May 11 '15 at 14:42

how to handle "Too many values to unpack"

5 Answers5