In Java, float can only store 32 bit numbers. It stores numbers as 1 bit for the sign (-/+), 8 bits for the Exponent, and 23 bits for the fraction.
Now, looking at your number and printint the decimal part,
System.out.println(Integer.toBinaryString(2123123));
results in 23 bit which means float does not have enough bits to store the fraction part and hence, it truncates the number.
To store the whole number, you need to use double which is 64 bit in Java, e.g.:
String stringAmount = "2123123.12";
double amount = Double.parseDouble(stringAmount);
System.out.println(amount);