Whats the difference between the java file found in jdk1.8.0_121/bin and the one that's in jdk1.8.0_121/jre/bin?
There is no difference in recent versions of Java; e.g. Java 8 or later. I have heard that there were small differences in older versions.
So the route for JAVA_HOME in the /etc/environment where should be pointing at?
The JAVA_HOME variable should point to the Java installation directory; i.e. jdk1.8.0_121 in your case.
If you are configuring the PATH variable to explicitly contain a Java bin directory1, you would normally use jdk1.8.0_121/bin so that the development tools are on the command search path.
Note that JAVA_HOME and PATH serve different purposes.
JAVA_HOME is used to tell 3rd-party tools where the Java installation is.
PATH is used by the shell to resolve command names.
1 - The other way to do it on Linux is to use the alternatives tool.