1

I am taking in a file which is encoded with one of two codecs:

try:
        decoded_file = open(fname, encoding ="ISO-8859-1")
except:
        decoded_file = open(fname, encoding ="utf-8")

What I am hoping to achieve is that if I get a UnicodeEncodeError I can switch to the appropriate codec

writer = csv.writer(decoded_file) Throws UnicodeEncodeError on utf-8 but not iso-8859-1

bobdawg
  • 69
  • 1
  • 8

1 Answers1

0

When you try to open a file with any encoding format ( though it is of diff encoding format) it does not throw any error. the error will be thrown when we read the file content (due to the given encoding format is incorrect and file cannot be read successfully).

so try something like this:

try:
    decoded_file = open(fname, encoding ="ISO-8859-1")
    writer = csv.writer(decoded_file)       
except:
    decoded_file = open(fname, encoding ="utf-8")
    writer = csv.writer(decoded_file)
Mahesh Anakali
  • 344
  • 1
  • 8
  • 1
    That still throws `UnicodeEncodeError: 'latin-1' codec can't encode character '\u039c' in position 3: ordinal not in range(256)` – bobdawg Sep 29 '20 at 04:56
  • Can you check this : (https://stackoverflow.com/questions/3942888/unicodeencodeerror-latin-1-codec-cant-encode-character) this might help you. – Mahesh Anakali Sep 29 '20 at 08:13
  • That does the trick, but i was hoping not to use that simply because it is "ignoring" any character that is unable to be encoded. – bobdawg Sep 29 '20 at 19:19