When you multiple chars together, they are promoted to int.
The int value of '3' is 51 (the ASCII value; as stated in JLS Sec 3.1, the first 128 characters of UTF-16 encoding are the same as ASCII characters).
So '3' * '3' * '3' == 51 * 51 * 51 == 132651.
But 132651 is too large to fit into a char when you cast it; so it overflows (twice, since Character.MAX_VALUE == 65535), and you get the value
(char) (c * c * c)
== (char) 132651
== (char) (132651 % 65536)
== (char) 1579
== 'ث'
But then you are assigning the result of the call to cube to an int (or long) variable called result; so it is promoted again to int, and then concatenated to the string: this means that the integer value is appended to the string, not the char. Hence:
Cubed 'char' = 1579
in the output.
Notice that if you call without assigning to result:
System.out.println("Cubed 'char' = " + cube(ch));
then the output is
Cubed 'char' = ث
Ideone demo