LinkedHashMap is a good choice for a data structure where you want to be able to put and get entries with O(1) running time, but you also need the behavior of a LinkedList.  The internal hashing function is what allows you put and get entries with constant-time.
Here is how you use LinkedHashMap:
Map<String, Double> linkedHashMap = new LinkedHashMap<String, String>();
linkedHashMap.put("today", "Wednesday");
linkedHashMap.put("tomorrow", "Thursday");
String today = linkedHashMap.get("today"); // today is 'Wednesday'
There are several arguments against using a simple HashMap and maintaining a separate List for the insertion order.  First, if you go this route it means you will have to maintain 2 data structures instead of one.  This is error prone, and makes maintaining your code more difficult.  Second, if you have to make your data structure Thread-safe, this would be complex for 2 data structures.  On the other hand, if you use a LinkedHashMap you only would have to worry about making this single data structure thread-safe.
As for implementation details, when you do a put into a LinkedHashMap, the JVM will take your key and use a cryptographic mapping function to ultimately convert that key into a memory address where your value will be stored.  Doing a get with a given key will also use this mapping function to find the exact location in memory where the value be stored.  The entrySet() method returns a Set consisting of all the keys and values in the LinkedHashMap.  By definition, sets are not ordered.  The entrySet() is not guaranteed to be Thread-safe.