Does identity "return its argument"?
It depends what you mean by the argument. 
- If it is the evaluated expression in the function call form, then not always.
 
- If it is what the body of the function sees on the stack upon entry, then yes, it does.
 
The anomaly arises because of the way that Clojure calls functions. 
- Clojure functions are objects that comply with the 
IFn
interface. 
- A Clojure function call translates into one of the many 
invoke
methods - overloaded for arity - of the function object. 
- All of the 
invoke methods have Object parameters. 
The upshot of all this is that every Clojure function call translates its every argument into some kind of Object - an identity operation except on primitives, which are wrapped in the corresponding Java class: long into Long, and so on. 
So even the identity function, essentially (defn identity [x] x), does not return a primitive argument. It can't, because it never sees it. 
For example, let's consider the expression 
(inc 3)
The number 3 is surely a long. What type is (inc 3)? Let's ask Clojure:
(type (inc 3))
=> java.lang.Long
... a boxed Long object. 
Hang on, are we sure that 3 is a primitive long?: 
(type 3)
=> java.lang.Long
Aaaaaaagh! It's boxed too!
Not necessarily! You can't tell, because by the time the body of type sees 3, it is boxed, whether or not it was so to the reader/compiler. The Clojure documentation is silent on the point. It just says that numeric literals are generally represented as per Java. 
So - in general - it's the evaluation mechanism, not a particular function (such as identity) that's responsible for boxing primitive arguments. This is auto-boxing. 
What your example shows is that primitives are held as such, un-boxed, at least in let forms:
(let [x 1.0] (identical? x x)) ;=> false
(let [x (identity 1.0)] (identical? x x)) ;=> true
The fact that identical? is able to distinguish between the two boxings of 1.0 shows that it is held as a primitive double. (I've used an ordinary double, just to show that the behaviour is nothing to do with the special value Double/NaN). 
Now let's try putting the number in a var:
(def x 1.0)
(identical? x x) ;=> true
(let [x (identity x)] (identical? x x)) ;=> true
It's boxed. 
While we're here, auto-boxing is idempotent:
(identical? x (identity x)) ;=> true
The above adds little to what Alan Thompson's and Josh's answers and Alan Malloy's  and Lee's comments comprise. I just felt they had hooked and played the fish, without actually landing it.