It happens because Scala has no union types like type T = String | Long. Your T from List[T] should have some concrete type, like String, Long or something, and you can't create list of several types. Every element in the list must have same T type, which makes many operations (like flatMap or filter) to be possible:
 scala> val a: (Int, Int, String, String, Int) = parseSomething() // tuple may be seen as List with heterogenous elements
 a: (Int, Int, String, String, Int) = (5, 6, "a", "b", 7)
 scala> val b = a.filter(_ != 6) //how compiler would know that `a._2 == 6` to infer (Int, String, String, Int) ?
 b: (???) = (5, "a", "b", 7)
So T should be same for all elements. When the compiler sees several parameters that have several types, it's trying to find the nearest common type:
def ff[T](a: T, b: T): T = a
trait A
case class A1() extends A
case class A2() extends A 
scala> ff(A1(), A2())
res20: A = A1() //A is the nearest common type
Generics are a little bit more interesting. The compiler may infer existential type for them M[_ >: A1 with A2]:
scala> ff(new LongHolder(1), new StringHolder("a"))
res23: Holder[_ >: String with Long] = LongHolder@a5e201f
It means Holder[T] forSome {type T >: String with Long}, which means that Holder is defined for some T (such T should exist but not necessary for all holders), that >: String with Long. Simply saying compiler asks "Hey! We need some T that bigger than Int with Long here" (just imagine types as boxes to put something in, and compiler is just a "foreman")
Finally Any is smaller (closest) type to fit in (it fits as it can be bigger than String with Long), so result is becoming Any, as value itself can not be existential in Scala.
P.S. Shapeless HList actually does what you want.