"a".replaceAll("a*", "b")
First replaces a to b, then advances the pointer past the b. Then it matches the end of string, and replaces with b. Since it matched an empty string, it advances the pointer, falls out of the string, and finishes, resulting in bb.
"a".replaceAll("a*?", "b")
first matches the start of string and replaces with b. It doesn't match the a because ? in a*? means "non-greedy" (match as little as possible). Since it matched an empty string, it advances the pointer, skipping a. Then it matches the end of string, replaces with b and falls out of the string, resulting in bab. The end result is the same as if you did "a".replaceAll("", "b").