How do I detect the encoding of some text?

Question

How can I identify which format some text is encoded in, either ASCII, Unicode or UTF-8?

Answer 1

Short answer

There is no guaranteed way to detect the encoding of an arbitrary set of bytes.

---

Long answer

• How can I detect the encoding/codepage of a text file
• Java: How to determine the correct charset encoding of a stream
• How to reliably guess the encoding between MacRoman, CP1252, Latin1, UTF-8, and ASCII
• GuessEncoding - only works for UTF-8, UTF-16LE, UTF-16BE, and UTF-32 ☹
• ICU Charset Detector
• cpdetector, free java codepage detection
• JCharDet (Java port of Mozilla charset detector) _{^{ironically, that page does not render the apostrophe in "Mozilla's" correctly}}

Ripped from my answer here.

---

See also

• Typical suggestion: Joel on Software The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)
• "Unicode" is not an encoding. What's the difference between unicode and utf8?
• Also, you should probably read over how to ask questions on Stack Overflow: Asking Help