There is no problem using:
a = Hash.new { Array.new }
If I execute a[2] it will return [], because that's the default value. It's just telling me what it is, nothing more.
Because the default is given as a block, the default value is a new empty array each time a[x] is invoked. The important thing to understand is that it is merely an empty array, not tied to the hash a in any way. That is, a[2] only answers the question, "what is the default value?"; it doesn't do anything with it. (As shown in other answers, if the block were written differently, all sorts of wondrous things could be done before returning the default value.)
Note that Hash.new { Array.new }, Hash.new { [] } and Hash.new { |h,k| Array.new } are all equivalent.
In order to add the key 2 to a, a[2] must be an lvalue. You have that later in your example:
a[2] = []
but if you do that there's no point to having a default. What you want to do is set a[2] equal to the default value:
a[2] = a[2]
(Note that if, as in this case, the return value of the block does not depend on the key, the line above is equivalent to:
a[2] = a[123456]
provided there is no key 123456.)
That adds the key-value pair 2=>[] to the hash, so now
a[2].push 2
a[2].push 3
a #=> {2=>[2, 3]}
It make more sense, however, to do this in one step:
a[2] = a[2].push 2
If a does not have a key 2, a[2] on the right of the equality will equal the default value of an empty array, 2 will be pushed onto it and the value for key 2 will be set equal to [2].
If a already has a key 2, say a[2] = [3,4,5], a[2] on the right will be [3,4,5], 2 will be pushed onto that and a (if it has no keys other than 2) will be:
a #=> {2=>[3,4,5,2]]}