If you really need a map you can use a LinkedHashMap which is sorted by insertion order. To compare the values you can create your own comparator:
Comparator<String> stringAndNumberComparator = (s1, s2) -> {
    try {
        return Double.valueOf(s1).compareTo(Double.valueOf(s2));
    } catch (NumberFormatException e) {
        return s1.compareTo(s2);
    }
};
Now you can use a java stream to sort the entry set of your map and collect it back to a LinkedHashMap:
Map<Integer, String> sorted = map.entrySet().stream()
        .sorted(Map.Entry.comparingByValue(stringAndNumberComparator))
        .collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue, (s1, s2) -> s1, LinkedHashMap::new));
Alternatively you can use the comparator provided in the answer from Karol Dowbecki with Map.Entry.comparingByValue().
The result will be {2=1.1, 5=2.2, 4=12, 3=google, 1=mark};
If you just need a list of your values you can just use this:
List<String> sorted = map.values().stream()
        .sorted(stringAndNumberComparator)
        .collect(Collectors.toList());
The result in this case will be [1.1, 2.2, 12, google, mark].