Why can't python map larger lists?

Question

I am using python 2.7.

This code works:

def function(b,c):
    return (b,c)

print(list(map(function,*zip([1]*2,[4]*2))))

However, this doesn't:

def function(b,c):
    return (b,c)

print(list(map(function,*zip([1]*20,[4]*20))))

What can be done to make the second snippet work?

At some point you're trying to put too much stuff in memory. Perhaps you could use an iterator, or a library designed for handling large arrays, or ...? What are you *actually* trying to achieve, contrived example notwithstanding? — jonrsharpe, Nov 21 '16 at 22:04
Trying to thread. See: http://stackoverflow.com/questions/40729592/passing-multiple-arguments-to-pool-map-using-class-function — zthomas.nc, Nov 21 '16 at 22:06
How exactly is this related to threading, or to that example? — jonrsharpe, Nov 21 '16 at 22:07
Superfluous `*zip()`. Try `print(list(map(function,[1]*20,[4]*20)))` — Steven Rumbalski, Nov 21 '16 at 22:08
Trying to figure out the best way to pass the arguments into `pool.map(func, args)` — zthomas.nc, Nov 21 '16 at 22:08

score 1 · Answer 1 · answered Nov 21 '16 at 22:09

You've gotten mixed up about what *zip([1]*20,[4]*20) does. It looks like you're trying to perform the following calls:

function(1, 4)
function(1, 4)
... [20 calls]

but that's not what you're actually doing. Your code actually tries to perform the following calls:

function(1, 1, 1, ... [20 1s])
function(4, 4, 4, ... [20 4s])

Your options are to stop using * and zip, change how function takes arguments, or use itertools.starmap:

def function(b, c):
    ...

print(list(map(function, [1]*20, [4]*20)))

or

def function(x):
    b, c = x
    ...

print(list(map(function, zip([1]*20, [4]*20))))

or

import itertools

def function(b, c):
    ...

print(list(itertools.starmap(function, zip([1]*20, [4]*20))))

1 Answers1