Numpy is observing the first element to see what dtype the array is going to have.  For a it sees a list and therefore produces an object array.  It happily moves on to fill in the rest of the elements into the object array.  For b, it sees a numeric value and assumes it's going to be some numeric dtype.  Then it borks when it gets to a list.
You can override this by stating object dtype in the first place
a=np.array([[1],2,3])
b=np.array([1,2,[3]], 'object')
print(a, b, sep='\n\n')
[list([1]) 2 3]
[1 2 list([3])]
Mind you, that may not be exactly how Numpy is identifying dtype but it's got to be pretty close.