0

I have the following script (test.py), printin a non-ASCII character:

# -*- coding: utf-8 -*-
print u"café"

It works as expected when running at shell:

$ python test.py 
café

However, if I try the usual bash redirect with > it fails in the following way:

$ python test.py > result.txt
Traceback (most recent call last):
  File "test.py", line 2, in <module>
    print u"café"
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 3: ordinal not in range(128)

How this can be solved?

Python version, in the case it helps:

$ python --version
Python 2.6.6

EDIT1: note this question is not a duplicated of this other. In the referred question they are using file abstraction in Python, e.g. open(), etc. while in my question these abstractions are not used and I'm using bash redirection.

EDIT2: check bash unicode support, as suggested in comments:

$ bash --version | head -n 1
GNU bash, versión 4.1.2(1)-release (x86_64-redhat-linux-gnu)
$ stty -a
speed 38400 baud; rows 35; columns 150; line = 0;
intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = M-^?; eol2 = M-^?; swtch = <undef>; start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R;
werase = ^W; lnext = ^V; flush = ^O; min = 1; time = 0;
-parenb -parodd cs8 -hupcl -cstopb cread -clocal -crtscts -cdtrdsr
-ignbrk -brkint -ignpar -parmrk -inpck -istrip -inlcr -igncr icrnl ixon -ixoff -iuclc ixany imaxbel -iutf8
opost -olcuc -ocrnl onlcr -onocr -onlret -ofill -ofdel nl0 cr0 tab0 bs0 vt0 ff0
isig icanon iexten echo echoe echok -echonl -noflsh -xcase -tostop -echoprt echoctl echoke
$ echo -e '\xe2\x82\xac'
€
fgalan
  • 11,732
  • 9
  • 46
  • 89
  • Your script (.py) file use the UTF-8 codepage ? Not ANSI or another codepage ? You need UTF-8 codepage for your python script file. – s3n0 May 22 '18 at 16:29
  • @William_Wilson I don't think is a duplicated. In the referred question they are using file abstraction in Python, e.g. open(), etc. while in my question these abstractions are not used and I'm using bash redirection. – fgalan May 22 '18 at 16:31
  • @s3n0 I have run `file test.py` and I get `test.py: UTF-8 Unicode text`. Not sure if you refer to that with "UTF-8 codepage". Otherwise, please tell me how to get the codepage and I'll provide the result. – fgalan May 22 '18 at 16:33
  • What about `bash --version` and terminal `stty -a` ? Is there a utf-8 support ? Try `echo -e '\xe2\x82\xac'`, when your terminal support utf-8, then you will see the result: `€`. – s3n0 May 22 '18 at 16:49
  • @s3n0 question post edited to included the detail you request. Check was ok. – fgalan May 22 '18 at 17:14
  • Here's the explanation for the problem - https://pythonhosted.org/kitchen/unicode-frustrations.html#frustration-3-inconsistent-treatment-of-output - with your `print u'café'` example too :). Did you find it here ? and can not you read it ? or do you just need to explain it better ? OMG ? :) – s3n0 May 22 '18 at 18:19
  • I use Python 2.7.13 and I do not have these problems. All the similar problems that can be found on the Internet, point to a problem with Python 2.6.x. Can you try a new version of Python 2.7.x (if possible)? UTF-8 support in Bash and Terminal are OK. So the problem will be apparent in Python - when converting Unicode to String (byte) when writing data to a text file. Unfortunately, if you do not have Python 2.7, you have to solve the problem with a more complex code entry, as shown on the web site above. – s3n0 May 22 '18 at 19:00
  • @s3n0 "café" was a random word I chose for the example, in Spanish but international enough to not sound too weird for English speakers. Probably the author of that article had a similar idea to chose the example. Funny :D – fgalan May 22 '18 at 19:46

0 Answers0