math in awk - unexpected formatting of floating-point numbers - loss of precision

Question

I am trying to do some simple math in awk

user@lab-client:~$ awk '{ram=(1.8 * 1024) * 1024; print ram}'

1.88744e+06

So I assume this means that this number is too large to be stored in variable "ram"

The total number is: 1887436.8

Lets try to store that number in the variable

user@lab-client:~$ awk '{ram=1887436.8; print ram}'

1.88744e+06

Same again. But what if we get rid of the "."?

user@lab-client:~$ awk '{ram=18874368; print ram}'

18874368

Further tests show that when the dot is in the number, it cannot be longer than 6 digits

user@lab-client:~$ awk '{ram=188743.68; print ram}'

188744

So its not a too large number, it is the dot that messes things up. How can I get around this?

Answer 1

you can control the number of decimal points with printf, eventually though the numbers won't be significant due to floating point representation

for example

awk 'BEGIN{for(i=5;i<20;i++) printf "%."i"f\n", 1./3}'

0.33333
0.333333
0.3333333
0.33333333
0.333333333
0.3333333333
0.33333333333
0.333333333333
0.3333333333333
0.33333333333333
0.333333333333333
0.3333333333333333
0.33333333333333331
0.333333333333333315
0.3333333333333333148

Answer 2

To complement karakfa's helpful answer:

In addition to explicit number formatting with printf and sprintf, you can also change awk's default floating-point number formatting, via built-in variable OFMT.
Note that it does not apply to integers.

OFMT defaults to %.6g, which means that any floating-point number is rounded to 6 significant digits and for exponents starting with 7 is represented in scientific notation.

Calculation result 1887436.8 - which has 8 significant digits - is therefore represented as 1.88744e+06, i.e., in scientific notation with 6 significant digits.

The following example sets OFMT to %1.f in order to output all floating-point numbers with 1 decimal place by default:

$ awk -v OFMT='%.1f' 'BEGIN {ram=(1.8 * 1024) * 1024; print ram}'
1887436.8

Note, however, that OFMT does not apply in the following scenarios:

• If the floating-point number is used in a string concatenation:

  $ awk -v OFMT='%.1f' 'BEGIN { print "result: " 1 / 3 }'
  result: 0.333333
  
  # Workaround: Use `sprintf()` with OFMT
  awk -v OFMT='%.1f' 'BEGIN { print "result: " sprintf(OFMT, 1 / 3) }'
  result: 0.3
  

• If a literal can be parsed as an integer - even if it looks like a floating-point number:

  $ awk -v OFMT='%.1f' 'BEGIN { print 1.000 }'
  1
  

Caveat: There are many subtleties around number conversion and formatting in awk, not least because of the limited precision of floating-point numbers (which in awk are always of the ISO C double type).