2

I need to encode a message from request and write it into a file. Currently I am using the URLEncoder.encode() method for encoding. But it is not giving the expected result for special characters in French and Dutch.

I have tried using URLEncoder.encode("msg", "UTF-8") also.

Example:
Original message: Pour gérer votre GSM
After encoding: Pour g?rer votre GSM

Can any one tell me which method I can use for this purpose?

John Kugelman
  • 349,597
  • 67
  • 533
  • 578
  • What are you trying to do exactly? Can you post some code snippet? URLEncoder.encode() is used to encode URLs (e.g., = to %3D), not really to perform character encoding. – Jack Leow Jul 01 '09 at 04:45

7 Answers7

2

URL encoding is not the right thing to do to preserve UTF-8 characters. See

What character set should I assume the encoded characters in a URL to be in?

Community
  • 1
  • 1
Erich Kitzmueller
  • 36,381
  • 5
  • 80
  • 102
1

Try doing something like:

BufferedWriter out = new BufferedWriter(new OutputStreamWriter(
                                        new FileOutputStream(file),"UTF-8"));
A_M
  • 7,693
  • 6
  • 33
  • 37
0

Have you tried using specifying OutputStream encoder using the [OutputStreamWriter(OutputStream, Charset)](http://java.sun.com/javase/6/docs/api/java/io/OutputStreamWriter.html#OutputStreamWriter(java.io.OutputStream,%20java.nio.charset.Charset)

notnoop
  • 58,763
  • 21
  • 123
  • 144
0

There are a lot of causes for the problem you have observed. The primary cause is that REQUEST is not giving you UTF-8 in the first place. I imagine that this situation will change over time, but currently there are many weak links that could be to blame: neither mySQL nor PHP5, html nor browsers use UTF-8 by default, though the data may originally be.

See stackoverflow: how-do-i-set-character-encoding-to-utf-8-for-default-html

and java.sun.com: technicalArticles--HTTPCharset

I experienced this problem with Chinese, and for that I'd recommend herongyang.com

Community
  • 1
  • 1
Tom Anderson
  • 942
  • 8
  • 16
0

I seems to me like every single web developer in the world stumbles over this. I'd like to point to an article that helped me a lot:

http://java.sun.com/developer/technicalArticles/Intl/HTTPCharset/

And if you use db2: this IBM developer works Article

By the way, I think the browsers don't support Unicode in addresses, because one could easily set up a phishing page when you use characters from one language that look similar to characters in another language.

BenMorel
  • 34,448
  • 50
  • 182
  • 322
Tim Büthe
  • 62,884
  • 17
  • 92
  • 129
0

if you are using tomcat then please see my post on the subject here http://nirlevy.blogspot.com/2009/02/utf8-and-hebrew-in-tomcat.html

I had the problem with hebrew but it's the same for every non english language

Nir Levy
  • 4,613
  • 2
  • 34
  • 47
0

Use an explicit encoding when creating the string you want to send:

final String input = ...;
final String utf8 = new String( input.getBytes( "UTF-8" ) , "UTF-8" );
Daniel Hiller
  • 3,415
  • 3
  • 23
  • 33
  • You can't choose (or change) the encoding of a string in Java. Character encodings only come into play when you convert between strings and other media, like writing to a file--and for that you should use an OutputStreamWriter like others have suggested. `new String(input.getBytes("UTF-8"), "UTF-8")` is just an expensive no-op. – Alan Moore Jul 01 '09 at 14:44
  • @Alan M: Hmm, I will check that out. We've had some encoding problems when the default character set on the platform was ISO-8859-1/15 so we inserted this statement to fix it. If you're saying this is a no-op it won't do any harm if we remove it, right? – Daniel Hiller Jul 02 '09 at 05:27
  • Right. Note that if you were doing that with a different encoding, like ISO-8859-1, you could actually corrupt the data. Any characters that weren't covered by that encoding would be replaced with junk in the encoding phase, and decoding it again would not recover them. But UTF-8 can handle any known character, so all you're doing is wasting clock cycles. – Alan Moore Jul 03 '09 at 01:38