Firstly curriedAdd is same as add2 _ and not add2. add2 is just a method. 
scala> curriedAdd
res52: Int => (Int => Int) = <function1>
scala> add2 _
res53: Int => (Int => Int) = <function1>
About the second question. I think the below is the reason. Doing
scala> val i = curriedAdd(23)
i: Int => Int = <function1>
scala> i _
res54: () => Int => Int = <function0>
scala> curriedAdd(23) _
<console>:10: error: _ must follow method; cannot follow Int => Int
              curriedAdd(23) _
curriedAdd(23) _ does not work. Lets look at scala manual (§6.7)-
The expression e _ is well-formed if e is of method type or if e is a
  call-by-name parameter. If e is a method with parameters, e _
  represents e converted to a function type by eta expansion (§6.26.5).
  If e is a parameterless method or call-by-name parameter of type =>T ,
  e _ represents the function of type () => T , which evaluates e when
  it is applied to the empty parameterlist ().
Remember it only evaluates if it is a method or call-by-name parameter. In  curriedAdd(23) _, it does not evaluate curriedAdd(23) but checks if it is a method or call-by-name. It is not a method nor a call-by-name parameter. 
It is not by-name because by-name is the property of variable. Above you get a by-name parameter after evaluating curriedAdd(23) but curriedAdd(23) in itself is not a by-name variable. Hence the error (Ideally the compiler should have coverted it). Note that the below works:
scala> curriedAdd(23)
res80: Int => Int = <function1>
scala> res80 _
res81: () => Int => Int = <function0>
The above works because res80 _, here you are applying _ to a call-by-name parameter and hence does the conversion.