Unicode to string conversion in Java

Question

I am building a language, a toy language. The syntax \#0061 is supposed to convert the given Unicode to an character:

String temp = yytext().subtring(2);

Then after that try to append '\u' to the string, I noticed that generated an error.

I also tried to "\\" + "u" + temp; this way does not do any conversion.

I am basically trying to convert Unicode to a character by supplying only '0061' to a method, help.

Note that 16 bits (4 hex digits) is not enough to represent all characters in Unicode. In java "\u1234" maps to a code point unit in UTF-16, which is not the same as a character. — Christoffer Hammarström, Nov 01 '10 at 15:06
Addendum: In fact it's the java data type char that maps to UTF-16 code point units, not actual Unicode characters. — Christoffer Hammarström, Nov 01 '10 at 15:16

Stephen C · Accepted Answer · 2009-12-20T04:42:21.563

11

Strip the '#' and use Integer.parseInt("0061", 16) to convert the hex digits to an int. Then cast to a char.

(If you had implemented the lexer by hand, an alternatively would be to do the conversion on the fly as your lexer matches the unicode literal. But on rereading the question, I see that you are using a lexer generator ... good move!)

edited Dec 20 '09 at 04:42

answered Dec 20 '09 at 04:33

Stephen C

698,415
94
811
1,216

1

Just curious: how did you spot that he's using a lexer? – BalusC Dec 20 '09 at 04:47
1

@BalusC Because of `yytext`, a lex specific variable – Pascal Thivent Dec 20 '09 at 05:02

BalusC · Answer 2 · 2009-12-20T04:46:21.757

2

You need to convert the particular codepoint to a char. You can do that with a little help of regex:

String string = "blah #0061 blah";

Matcher matcher = Pattern.compile("\\#((?i)[0-9a-f]{4})").matcher(string);
while (matcher.find()) {
    int codepoint = Integer.valueOf(matcher.group(1), 16);
    string = string.replaceAll(matcher.group(0), String.valueOf((char) codepoint));
}

System.out.println(string); // blah a blah

Edit as per the comments, if it is a single token, then just do:

String string = "0061";
char c = (char) Integer.parseInt(string, 16);
System.out.println(c); // a

edited Dec 20 '09 at 04:46

answered Dec 20 '09 at 04:34

BalusC

1,082,665
372
3,610
3,555

Erm ... you don't want to implement a lexical analyser using Java regex pattern matching. – Stephen C Dec 20 '09 at 04:35
I need something like the first example you posted. I ran the code making the pattern changes as I need them however the ReplaceAll doesn't replace anything. The string is the same as the original string :( – May 22 '12 at 13:16
2

@Eric: press `Ask Question` button on right top to ask a question on which you would like to get answers. – BalusC May 22 '12 at 13:19

score 2 · Answer 3 · answered Dec 20 '09 at 04:45

i am basically trying to convert unicode to a character by supplying only '0061' to a method, help.

char fromUnicode(String codePoint) {
  return (char)  Integer.parseInt(codePoint, 16);
}

You need to handle bad inputs and such, but that will work otherwise.

score 0 · Answer 4 · answered Dec 20 '09 at 04:30

\uXXXX is an escape sequence. Before execution it has already been converted into the actual character value, its not "evaluated" in anyway at runtime.

What you probably want to do is define a mapping from your #XXXX syntax to Unicode code points and cast them to char.

Unicode to string conversion in Java

4 Answers4

Linked