What are rules governing instantiating instances of generic outer class and its inner generic inner class and declaring respective references?

Question

1) Why obj4, obj6, obj7 give compile error, and obj5 is fine? Where can I read about rules, regulating using such generic coupled outer-inner classes? I found nothing directly to the point.

When I don't supply any type argument for Inner class in obj3 it is fine (while Inner class needs to be given something for for its fld3 field of type S), but when I do the same and do not supply type for Outer class - it does not compile (obj4) - though Object could be implied...

2) Besides, why obj10 compiles fine, but obj11 fails? In obj10 line I also write outerInstance.new Inner<>(), implying for Inner that S is Object. But this is not problem for inner, but same "trick" is problem for Outer...

        //in Class A of package com.sth
        public class MyGen<T> {
            T fld1;

            class GenInner<S> {
                T fld2;
                S fld3;
            }

                // within main of Class Driver of same package com.sth
                MyGen.GenInner obj1 = new MyGen<String>().new GenInner<Integer>();
                MyGen.GenInner obj2 = new MyGen<String>().new GenInner<>();
                MyGen.GenInner obj3 = new MyGen<String>().new GenInner();
                //MyGen.GenInner obj4 = new MyGen().new GenInner<String>();  //ERR !
                MyGen.GenInner obj5 = new MyGen<>().new GenInner<String>();

                //MyGen<String>.GenInner obj6;  // ERR
                //MyGen.GenInner<String> obj7;  // ERR
                MyGen<String>.GenInner<Integer> obj8;
                MyGen.GenInner obj9;     


  MyGen<String>.GenInner<Integer> obj10 = new MyGen<String>().new GenInner<>();

  //Type mismatch: cannot convert from MyGen<Object>.GenInner<Integer> to MyGen<String>.GenInner<Integer>
  //MyGen<String>.GenInner<Integer> obj11 = new MyGen<>().new GenInner<Integer>();  // ERR!

These answers relate to my question, but do not provide any clue:

1. Answer 1
2. Answer 2
3. Answer 3
4. Answer 4
5. Answer 5

Answer 1

The examples which don't compile are mostly examples of rare types. (Also, as noted by John in the comments, the obj3 example shouldn't compile either.)

A raw type is the type which is formed by using a generic type without an accompanying type argument list (like e.g. Set, as opposed to e.g. Set<Float>). A rare type is when you have a generic outer class and generic inner class, one of which is raw and the other is not.

Taken from JLS 4.8 Raw Types:

More precisely, a raw type is defined to be one of:

• The reference type that is formed by taking the name of a generic type declaration without an accompanying type argument list.

• An array type whose element type is a raw type.

• A non-static member type of a raw type R that is not inherited from a superclass or superinterface of R.

(Note that the point in bold implies that if you have a raw type MyGen, then its non-static member class GenInner must also be raw, so there is no such thing as e.g. a MyGen.GenInner<String>.)

Another implication of the rules above is that a generic inner class of a raw type can itself only be used as a raw type:

class Outer<T>{
    class Inner<S> {
        S s;
    }
}

It is not possible to access Inner as a partially raw type (a "rare" type):

Outer.Inner<Double> x = null;  // illegal
Double d = x.s;

because Outer itself is raw, hence so are all its inner classes including Inner, and so it is not possible to pass any type arguments to Inner.

It is a compile-time error to pass type arguments to a non-static type member of a raw type that is not inherited from its superclasses or superinterfaces.

It is a compile-time error to attempt to use a type member of a parameterized type as a raw type.

This means that the ban on "rare" types extends to the case where the qualifying type is parameterized, but we attempt to use the inner class as a raw type:

Outer<Integer>.Inner x = null; // illegal

This is the opposite of the case discussed above. There is no practical justification for this half-baked type. In legacy code, no type arguments are used. In non-legacy code, we should use the generic types correctly and pass all the required type arguments.

The example with obj11 = new MyGen<>().new GenInner<Integer>() isn't a rare type. It just looks like a regular failure of type inference with the diamond, because the expression new MyGen<>() isn't assigned to anything. In absence of an assignment, Object is typically assumed. (More technically it would be whatever the upper bound on the type variable is, which is Object in this case.)

---

Also, while not directly related to the question at hand, the following are the only two forms which should actually be used in new code:

MyGen<String>.GenInner<Integer> ok1 = new MyGen<String>().new GenInner<Integer>();
MyGen<String>.GenInner<Integer> ok2 = new MyGen<String>().new GenInner<>();

All of the others (which compile) use raw types, and raw types are discouraged.