Use of yield with a dict comprehension

Question

As a contrived example:

myset = set(['a', 'b', 'c', 'd'])
mydict = {item: (yield ''.join([item, 's'])) for item in myset}

and list(mydict) gives:

['as', 'cs', 'bs', 'ds', {'a': None, 'b': None, 'c': None, 'd': None}]

What happens here? What does yield do? And is this behavior consistent no matter what expression follows yield?

Note: I know that doing mydict = {item: ''.join([item, 's']) for item in myset} would give the dictionary {'a': 'as', 'b': 'bs', 'c': 'cs', 'd': 'ds'}, which seems to be what I am trying to do here.

Wow. I would've lost good money betting that was a SyntaxError. — DSM, Sep 10 '12 at 19:22
@DSM -- Me too. I'm guessing that the dict comprehension is implemented as a wrapper around generator expressions. (after all, `{k:v for k,v in ...}` is equivalent to `dict((k,v) for k,v in ...)` — mgilson, Sep 10 '12 at 19:28
I think the moral of the story is: "Don't try this at home" ;-). But very interesting question (+1). — mgilson, Sep 10 '12 at 19:42
In python3 the `list(mydict)` is just `['as', 'cs', 'bs', 'ds']` — wim, Mar 14 '14 at 01:07

score 12 · Accepted Answer · answered Sep 10 '12 at 19:34

First of all, what does yield return? The answer in this case is None, because yield returns the parameter passed to next(), which is nothing in this case (list doesn't pass anything to next).

Now here's your answer:

>>> myset = set(['a', 'b', 'c', 'd'])
>>> mydict = {item: (yield ''.join([item, 's'])) for item in myset}
>>> mydict
<generator object <dictcomp> at 0x0222BB20>

The dict comprehension is turned into a generator, because you used yield in a function body context! This means that the whole thing isn't evaluated until it's passed into list.

So here's what happens:

list calls next(mydict).
Yield returns ''.join([item, 's']) to list and freezes the comprehension.
list calls next(mydict).
The comprehension resumes and assigns the result of yield (None) to item in the dictionary and starts a new comprehension iteration.
Go back to 1.

And at last the actual generator object returns the temporary in the body, which was the dict. Why this happens is unknown to me, and it's probably not documented behaviour either.

score 4 · Answer 2 · edited May 23 '17 at 11:54

4

I think that yield is turning your nice dictionary comprehension into a generator expression. So, when you're iterating over the generator, yield is "yielding" the elements that look as, bs ..., but the statement yield ... returns None. So, at the end of the day, you get a dictionary looking like {'a': None, 'b': None, ...}.

The part that confuses me is why the dictionary is actually yielded at the end. I'm guessing that this behavior is actually not well defined by the standard, but I could be wrong about that.

Interestingly enough, if you try this with a list comprehension, python complains:

>>> a = [(yield i) for i in myset]
  File "<stdin>", line 1
SyntaxError: 'yield' outside function

But it's Ok in a generator (apparently).

edited May 23 '17 at 11:54

Community

1
1

answered Sep 10 '12 at 19:23

mgilson

300,191
65
633
696

3

FWIW pypy doesn't give the dictionary: `list(mydict)` produces `['as', 'cs', 'bs', 'ds']`, which if you'd told me that the expression was valid would be what I'd've guessed it would have returned. – DSM Sep 10 '12 at 19:29
2

So either pypy has it wrong, or it isn't defined behavior (my money's on the second). – mgilson Sep 10 '12 at 19:31
@DSM -- What does `pypy` return for `dict( (item,(yield (item+'s'))) for item in 'abdc')`? – mgilson Sep 10 '12 at 19:35
`{'a': None, 'c': None, 'b': None, 'd': None}`. – DSM Sep 10 '12 at 19:37
@DSM -- Thanks. Not sure where I was going with that thought, but, it's interesting (and not surprising) that pypy and Cpython agree on that one. – mgilson Sep 10 '12 at 19:41

defuz · Answer 3 · 2012-09-10T22:19:43.287

I find! ^_^

In normal life, expression

print {item: (yield ''.join([item, 's'])) for item in myset}

evaluate like this:

def d(myset):
    result = {}
    for item in myset:
        result[item] = (''.join([item, 's']))
    yield result

print d(myset).next()

Why yield result instead return result? I think it is necessary to support nested list comprehensions* like this:

print {i: f.lower() for i in nums for f in fruit}  # yes, it's works

So, would look like this code?

def d(myset):
    result = {}
    for item in myset:
        result[item] = (yield ''.join([item, 's']))
    yield result

and

>>> print list(d(myset))
['as', 'cs', 'bs', 'ds', {'a': None, 'b': None, 'c': None, 'd': None}]

First will be returned all values of ''.join([item, 's']) and the last will be returned dict result. Value of yield expression is None, so values in the result is None too.

* More correct interpretation of evaluate nested list comprehensions:

print {i: f.lower() for i in nums for f in fruit}

# eval like this:

result = {}
for i, f in product(nums, fruit): # product from itertools
    key, value = (i, f.lower())
    result[key] = value
print result

score 0 · Answer 4 · answered Sep 10 '12 at 21:09

I think that your code has to execute similarity of this:

def d(myset):
    for item in myset:
        yield item, (yield ''.join([item, 's']))

d(myset)

Firstly, evaluated yield ''.join([item, 's'] (and return 'as', 'cs', etc.). Value of yield expression is None, because is sent back to the generator. And then eval yield item, None, that return tuples ('a', None), ('b', None).

So, I have:

>>> list(d(myset))
['as', ('a', None), 'cs', ('c', None), 'bs', ('b', None), 'ds', ('d', None)]

What happens next, I do not understand.

Use of yield with a dict comprehension

4 Answers4

Linked