Some distros, by default, reset your environment variables to ensure programs executed under root run as expected. Any environment variable not specified in a whitelist is not carried into your root session.
The instruction to reset to environment as well as what variables to allow is defined in /etc/sudoers. (You need permissions to view/edit this).
For example, on my Fedora 16 box, I have
Defaults env_reset
Defaults env_keep = "COLORS DISPLAY HOSTNAME HISTSIZE INPUTRC KDEDIR LS_COLORS"
Defaults env_keep += "MAIL PS1 PS2 QTDIR USERNAME LANG LC_ADDRESS LC_CTYPE"
Defaults env_keep += "LC_COLLATE LC_IDENTIFICATION LC_MEASUREMENT LC_MESSAGES"
Defaults env_keep += "LC_MONETARY LC_NAME LC_NUMERIC LC_PAPER LC_TELEPHONE"
Defaults env_keep += "LC_TIME LC_ALL LANGUAGE LINGUAS _XKB_CHARSET XAUTHORITY"
Note that XAUTHORITY isn't there by default, I had to add it.
If you want your JAVA_HOME envvar to carry over, you could add it here. However, for this to be all you need to do, you'd need to add PATH to this list and that is really discouraged.
So also in this file is a line like the following:
Defaults secure_path = /sbin:/bin:/usr/sbin:/usr/bin
This will be your $PATH inside your sudo session. Just ensure that you have a symlink to the java executable you want to use in one of those folders, and you'll be fine.
(Since you are getting the "command not found message", there are no java executables on your secure path, but if there were, you'd need to either remove it, or place your new symlink "higher" (earlier) in the path)
Alternative 1
Add an alias to java (eg alias java='/usr/java/latest/bin/java') inside a file like /etc/bashrc
Alternative 2
Add an alias to java in your local profile or bashrc files and an alias to sudo that will preserve your aliases.