Cannot reshape array of size into shape

Question

I am following a machine learning video on youtube at https://www.youtube.com/watch?v=lbFEZAXzk0g. The tutorial is in python2 so I need to convert it into python3. Here is the section of the code I am having an error with:

def load_mnist_images(filename):
    if not os.path.exists(filename):
        download(filename)
    with gzip.open(filename,'rb') as file:
        data = numpy.frombuffer(file.read(),numpy.uint8, offset=16)
        data = data.reshape(-1,1,28,28)
        return data/numpy.float32(256)

I am getting this error: ValueError: cannot reshape array of size 9992 into shape (1,28,28). How do I fix this? In the tutorial it was working. Also, if I have any other errors please tell me.

Answer 1

Your input does not have the same number of elements as your output array. Your input is size 9992. Your output is size [? x 1 x 28 x 28] since the -1 indicates that the reshape command should determine how many indices along this dimension are necessary to fit your array. 28x28x1 is 784, so any input you want to reshape to this size must be neatly divisible by 784 so it fits in the output shape. 9992 is not divisible by 784, so it is throwing a ValueError. Here is a minimal example to illustrate:

import numpy as np

data = np.zeros(2352) # 2352 is 784 x 3
out = data.reshape((-1,1,28,28)) # executes correctly -  out is size [3,1,28,28]

data = np.zeros(9992) # 9992 is 784 x 12.745 ... not integer divisible
out = data.reshape((-1,1,28,28)) # throws ValueError: cannot reshape array of size 9992 into shape (1,28,28)

So, if you don't want a ValueError, you need to reshape the input into a differently sized array where it fits correctly.

Answer 2

the reshape has the following syntax

data.reshape(shape)

shapes are passed in the form of tuples (a, b). so try,

data.reshape((-1, 1, 28, 28))

Answer 3

Try like this

import numpy as np

x_train_reshaped=np.reshape(x_train,(60000, 28, 28))
x_test_reshaped=np.reshape(x_test,(10000, 28, 28))

Answer 4

I had similar problems but not with images, it looks similar in a way, so here is how I did mine You might need to resize the data first: the data in the code below is your size =9992

datas= np.array([data], order='C')
datas.resize((1,28,28))
datas.shape

Answer 5

You have to balance index and column labels e.g. 6x6=36

DataFrame(np.random.rand(36).reshape((6,9)),
index=['row1','row2','row3','row4','row5','row6',],
columns = ['colmn1','colmn2','colmn3','colmn4','colmn5','colmn6',])

or 4×9=36 would have:

DataFrame(np.random.rand(36).reshape((4,9)),
index=['row1','row2','row3','row4',],
columns = ['colmn1','colmn2','colmn3','colmn4','colmn5','colmn6','colmn7','colmn8','colmn9',])

Answer 6

image = cv2.imread("/content/paper1.jpg")
_shape_1= image.shape
print("The shape of image is:",_shape_1)
print("The product shape of image is:", np.prod(_shape_1))

dim = np.prod(_shape_1)
num = dim
factors=[]
for i in range(1,num+1):
    if num%i==0:
       factors.append(i)
#print ("Factors of shape is {}  = {}".format(num,factors))

test = []
for i in factors:
  for j in factors:
    for k in range(0,1000):
      if i*j*k == num:
        if i <= 3000 and j <= 5000 and k <=3: # set this values acc to req shape
          test.append([i,j,k])

print("The possible values to reshape: ",test)
print("The size to be reshaped to:",tuple(test[0]))        
moy = np.reshape(image,tuple(test[0])
new_shape = moy.shape
print("The reshaped image size is: ",new_shape)

code to find the all possible reshape values for the given image shape.

Answer 7

Your are returning the value inside a "with" block, that could be the problem In Python, if I return inside a "with" block, will the file still close? , and like psnbaba has already mentioned: you have to pass the shape as a tuple Try this:

def load_mnist_images(filename):
    if not os.path.exists(filename):
        download(filename)
    with gzip.open(filename,'rb') as file:
        data = numpy.frombuffer(file.read(),numpy.uint8, offset=16)
    data = data.reshape((-1,1,28,28))
    return data/numpy.float32(256)
 

If that doesn't work, then you might try this function:

def get_possible_2d_shapes(nparray):
    shape = nparray.shape
    dim = np.prod(shape) #in case nparray has more then one dimension
    rang = np.arange(dim)[1:-1] #no zero
    todevide = np.repeat(dim, rang.shape[0]) #create array to devide
    restresults = np.true_divide(todevide, rang) #prefilter with numpy -> faster
    good = list(dict.fromkeys(np.real_if_close(restresults).astype(np.int64).tolist())) #remove duplicates
    allresults = [] #check the numpy results with python to avoid errors
    for g in good:
        if dim % g == 0:
            allresults.append((g, dim // g))
    return allresults
print(onebytearray)
print(f'{onebytearray.shape=}')
allshapes=(get_possible_2d_shapes(nparray=onebytearray))
reshaped=onebytearray.reshape((-1,1,*allshapes[4]))
print(reshaped)
%timeit get_possible_2d_shapes(nparray=onebytearray)
[48 48 48 ... 32 63 10]
onebytearray.shape=(47654470,)
[[[[ 48  48  48 ...  48  32  32]
   [ 52 100  32 ...  48  32  48]
   [ 48  32  48 ...  32 102 102]
   ...
   [101  48  48 ...  32  32  32]
   [ 32  32  32 ...  32  32  32]
   [ 32  32  32 ...  32  63  10]]]]
1.53 s ± 5.46 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)