2

I have an array W containing float numbers. 

W.dtype = float32 
type(W) = <type 'numpy.ndarray'>

Then I pickle.dump() it into a mr.pkl file, 

pickle.dump(W, open("/home/mr.pkl", "wb"))

but when I load it,

pickle.load(open("/home/mr.pkl","rb"))

an error occurs:

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 1206: ordinal not in range(128)

I don't know why, I was confused about it for a week, can any one help me about this? any help is appreciated, thank you a lot!

Andrea Corbellini
  • 17,339
  • 3
  • 53
  • 69
Yunshi
  • 43
  • 2
  • 5

2 Answers2

8

I had the same problem, this code worked for me. The encoding='latin1' is the important part.

# read in data from pickle file created with Load_Data.py
pickle_file = 'mnist.pkl'
with open(pickle_file, 'rb') as f:
    train_set, valid_set, test_set = pickle.load(f, encoding='latin1')
Linda MacPhee-Cobb
  • 7,646
  • 3
  • 20
  • 18
  • thanks Linda! I think your method works when I load data in other language, but I think my problem is not this, because W only contains float number, so I found that I dump the data using python2.7 while load it by python3.5, so it's may because Pickle of these two version are not compatible, thank you for your kind reply again! – Yunshi Feb 12 '16 at 13:42
  • The MNIST data set is all floats and ints, but thanks, I'll try the dump data next time. – Linda MacPhee-Cobb Feb 12 '16 at 20:57
0

OK, I find out a solution but only in console mode.
The steps are as followings.

  1. Why and Can not import mnist_loader? Yes, you should put your 'yourscript.py' in the same folder with mnist_loader.py
  2. Why an error 'ascii' codec can't decode byte 0x90 in position 614: ordinal not in range(128)'? Yes, 'yourscript.py' need in the same folder with mnist_loader.py >> Like me if you use IPython, you need download yourscript.ipynb as 'yourscript.py'
  3. If uses P3.X, please rewrite something in mnist_loader.py as followings " import _pickle as Pickle" and "Pickle.load(f, encoding='latin1')"
  4. When you try to print or get data from Pickle.load(...), please transfer 'zip' type from output of Pickle.load(...) as a list type like list(train_data), ...
  5. In console mode, ...in mnist_loader.py folder(in general ..\src)...>> Python yourscript.py
  6. You can see it is workable. For example: print('len ', len(list(train_data))), it shows 50000.

But...

7. I still do not know why it is not workable in IPython.

october
  • 3
  • 2