Error reading a from file - Encoding issue

Question

Im reading a CSV file that was created from MS Excel. When I open it up in notepad it looks ok, but in Notepad++ I change the Encoding from ANSI to UTF8 and a few non printed characters turn up.

Specifically xFF. -(HEX Value)

In my C# app this character is causing an issue when reading the file so is there a way I can do a String.replace('xFF', ' '); on this?

Update

I found this link on SO, as it turns out it is the answer to my question but not my problem. Link

score 2 · Accepted Answer · answered Nov 27 '12 at 05:18

2

Instead of String.Replace, Specify encoding while reading the file.

Example

File.ReadAllText("test.csv",System.Text.UTF8Encoding)

answered Nov 27 '12 at 05:18

Tilak

30,108
19
83
131

Since I have no control over where the files come from, can the encoding be determined before reading ? – IEnumerable Nov 27 '12 at 05:22
1

How notepad does it -> http://blogs.msdn.com/b/oldnewthing/archive/2007/04/17/2158334.aspx – Tilak Nov 27 '12 at 05:28

score 1 · Answer 2 · answered Nov 27 '12 at 05:36

1

Guess your unicode representation is wrong. Try this

string foo = "foo\xff";
foo.Replace('\xff',' ');

answered Nov 27 '12 at 05:36

Antony Thomas

3,576
2
34
40

I'm not sure it's a good thing to "hack" the encoding. I think Tilak answer is better even if not perfect. Even better would be to know/understand what are all the encodings used in the OP's file transformation process. – Guillaume Nov 27 '12 at 06:18

Error reading a from file - Encoding issue

Update

2 Answers2