18

I am working on an application which is in Django. I am trying to store some data structure e.g. dictionary in MySQL db. So i use Python Pickle module. It works fine when i store it in db using pickle.dumps(some_structure). My DB field is longblob mode is binary.

But when i access the field of model object in django:

obj = someModel.get(pk=1)
some_structure = obj.field
content  = pickle.loads(some_structure)

it raises following error:

UnpicklingError: invalid load key, '{'.

Please help me, i have tried google but it does'nt help me, also there is one similar Question but it is not related to my problem as i am storing in db.

Community
  • 1
  • 1
Aamir Rind
  • 38,793
  • 23
  • 126
  • 164
  • I would suggest pickling your data to store it in the database is not the correct way to go about this. You should store your data in the database correctly, transforming your data into tables and fields. – Gareth Latty Nov 13 '11 at 11:21
  • The data is in `complex hierarchy (nested dictionaries)`. I can not save it in form of tables and fields. The data is result of some process which i later need to process in django on another `View`. – Aamir Rind Nov 13 '11 at 11:39
  • 1
    Then JSON is probably a better format than pickle. JSON is cross-platform, whereas pickle is implementation-specific. – Daniel Roseman Nov 13 '11 at 12:02
  • Anyway pickled data is not possible to filter. – Melug Nov 13 '11 at 12:18
  • What model field type do you use? – alex vasi Nov 13 '11 at 13:47
  • @alex i am using `Text` field. I have also tried to create my own `Binary Model Field` storing in `base64` but also got the same error. – Aamir Rind Nov 14 '11 at 12:42
  • If base64 encoding doesn't help, then probably the problem is somewhere in your "pickling" code. '{' looks like str(dict()) not a pickle.dumps(dict()). Plz, inspect data stored in database and compare it with manually pickled object in python REPL. Once again, try debugger: http://stackoverflow.com/questions/7638819/how-to-debug-with-django – alex vasi Nov 14 '11 at 18:42

2 Answers2

23

I encountered this error and I discovered that it was because I had tried to unpickle something that had not originally been pickled.

More specifically, I had stored a Python dict without pickling it, leading to the initial character {.

Thus, to solve the problem, you should try either:

  • Not unpickling the string upon retrieval from the datastore, or
  • Verifying that objects are being pickled prior to insertion into the datastore
einnocent
  • 3,567
  • 4
  • 32
  • 42
1

A corrupt file may cause this issue. Replace the older pickle object with new one. It worked for me.

Sunil Sharma
  • 249
  • 3
  • 8