Unmarshal json to float. Why is float64 required?

Question

I've noticed some strange behavior with the way go unmarshals json floats. Some numbers, but not all, refuse to unmarshal correctly. Fixing this is as easy as using a float64 instead of a float32 in the destination variable, but for the life of me I can't find a good reason why this is the case.

Here is code that demonstrates the problem:

package main

import (
    "encoding/json"
    "fmt"
    . "github.com/shopspring/decimal"
)

func main() {
    bytes, _ := json.Marshal(369.1368) // not every number is broken, but this one is
    fmt.Println("bytes", string(bytes))

    var f32 float32
    json.Unmarshal(bytes, &f32)
    fmt.Printf("f32 %f\n", f32) // adds an extra 0.00001 to the number

    var d Decimal
    json.Unmarshal(bytes, &d)
    fmt.Printf("d %s\n", d) // 3rd party packages work

    // naw, you can just float64
    var f64 float64
    json.Unmarshal(bytes, &f64)
    fmt.Printf("f64 %f\n", f64) // float64 works
}

A float64 isn't required to accurately represent my example number, so why is required here?

Go playground link: https://play.golang.org/p/tHkonQtZoCt

I haven't looked closely, but I'd say the cause is roundoff error due to the nature of floating-point numbers: the number whose decimal representation is `369.1368` may not be exactly representable by a `float32`. See https://stackoverflow.com/questions/588004/is-floating-point-math-broken — jub0bs, Oct 25 '19 at 22:06

score 4 · Accepted Answer · answered Oct 25 '19 at 22:51

Your assertion is wrong: 369.1368 cannot be represented exactly by either float32 or float64.

The closest float32 value is (approximately) 369.136810302734375, whcih rounds to 369.13681 which is where your extra digit comes from. The closest float64 value is (approximately) 369.13679999999999382, which rounds more nicely for your purposes.

(Of course, if you round either of these to just four digits after the decimal point, you get the number you expected.)

A Decimal representation is exact: there is no rounding error.

JSON transmits and receives floating point values expressed in decimal, but actual implementations, in various languages, then encode those numbers in different ways. Depending on what sort of entity you're talking to via JSON, encoding and decoding via Decimal could preserve the number exactly as you'd like, but be aware that programs written in, say, C++ or Python might decode your number to a different floating-point precision and introduce various rounding errors.

This Go Playground example uses the newly-added %x format, and shows you how the numbers are stored internally:

as float32 = 369.13681030273437500 (float32), which is really 12095875p-15 or 0x1.712306p+08

and:

as float64 = 369.13679999999999382 (float64), which is really 6493923261440380p-44 or 0x1.712305532617cp+08

That is, the number 369.whatever is internally represented in binary. It's between 2⁸ = 256 and 2⁹ = 512. In binary, it is 1 256, no 128, 1 64, 1 32, 1 16, no 8, no 4, no 2, and 1 1: 1.01110001something x 2⁸. The %b format expresses this one way and the %x format another, with %x starting with 1.72 (1 . 0111 0010).

See Is floating point math broken? (as jub0bs linked in a comment) for more.

So I guess the real answer is, the first line of the main function is using a float64 at compile time? My tests indicate that this would be true. — fishybell, Oct 26 '19 at 23:22
On further examination, forcing the variable to a float32 still allows for the json marshaler to correctly format the output: [https://play.golang.org/p/IRr2dxY4KvV](https://play.golang.org/p/IRr2dxY4KvV). My best guess at this time is that a) a lot of my constants are indeed poor choices for float32, and b) the json marshaler only writes 4 digits of precision without [explicitly forcing the formatting](https://play.golang.org/p/leGylce2aE2). — fishybell, Oct 26 '19 at 23:36
You can see the json unmarshaller source code [here](https://golang.org/src/encoding/json/decode.go?#L903) (scan down to the `default` case that handles number, and then the `reflect.Float32, reflect.Float64` case). You can see the default marshaller [here](https://golang.org/src/encoding/json/encode.go?#L545). These point directly to source line numbers that might change over time, though. — torek, Oct 27 '19 at 05:17
See also these two closed Golang issues: https://github.com/golang/go/issues/6384 https://github.com/golang/go/issues/14135. I suggest you read through the second one closely and follow some of the links to committee discussions on formatting numbers in JSON. — torek, Oct 27 '19 at 05:19

Unmarshal json to float. Why is float64 required?

1 Answers1