1

I have a script that reads an XML file and writes it into the Database.
When I run it through the browser (call it via a view) it works fine, but
when I created a Command for it (./manage.py importxmlfile) I get the following message:

UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 6: ordinal not in range(128)

I'm not sure why it would only happen when calling the import via command line.. any ideas?

Update
I'm trying to convert an lxml.etree._ElementUnicodeResult object to string and save it in the DB (utf8 collation) using str(result).
This produces the error mentioned above only on Command Line.

Andbdrew
  • 11,788
  • 4
  • 33
  • 37
Asaf
  • 8,106
  • 19
  • 66
  • 116
  • Can you post your source code? – Mounir Jun 09 '13 at 18:21
  • it's a pretty massive project, not really an option. – Asaf Jun 09 '13 at 18:24
  • odd, it works when I debug it on Eclipse, but with Konsole it doesn't – Asaf Jun 09 '13 at 18:29
  • 1
    Try to add `# -*- coding: utf-8 -*-` on bottom of your command file. And for your objects's unicode methods try to return unicoded strings: `return u"%s" %(self.param)` – Mounir Jun 09 '13 at 18:31
  • doesn't seem to have an effect – Asaf Jun 09 '13 at 18:35
  • Sorry I can't help more without code. good luck – Mounir Jun 09 '13 at 18:37
  • Be careful messing with your system or python encoding environment variables in an attempt to fix this sort of problem. It will likely result in your script running fine on your computer but then breaking when you deploy or run the script somewhere else. – Andbdrew Jun 09 '13 at 19:05

1 Answers1

1

Ah, don't use str(result).

instead, do:

result.encode('utf-8')

When you call str(result), python will use the default system encoding (usually ascii) to try and encode the bytes in result. This will break if the ordinal not in range(128). Rather than using the ascii codec, just .encode() and tell python which codec to use.

Check out the Python Unicode HowTo for more information. You might also want to check out this related question or this excellent presentation on the subject.

Community
  • 1
  • 1
Andbdrew
  • 11,788
  • 4
  • 33
  • 37
  • I had to make sure to use it when using regex (on an external function), and on some other pre-save functions, but worked like a charm. – Asaf Jun 09 '13 at 19:08