There is a difference, although it's not intuitive. I recommend reading the java standard tutorial for more details.
Basically, the two are different and incompatible types. List<E> can store objects that extend, subclass, or actually are type E, but it's its own type. You cannot refer to a List<String> from a List<Object> reference, even though you can add a String to a List<Object>. 
List<?> means that it's a list reference that can refer to any parameterized reference of it. A List<?> can refer to a List<String> or a List<Integer>. It's most useful when it's bounded by some interface or class. For example, the first method below (adapted from the java standard tutorial) will only take List<Number>, and not anything like List<Double> or List<Integer>.
public static double sumOfList(List<Number> list) {
    double s = 0.0;
    for (Number n : list)
        s += n.doubleValue();
    return s;
}
But the following code using wildcards can take List<Double> or List<Integer>. It's more flexible.
public static double sumOfList(List<? extends Number> list) {
    double s = 0.0;
    for (Number n : list)
        s += n.doubleValue();
    return s;
}