Hi all,
I have 2 large int64 values, A and B. One of them, namely B, is close to math.MaxInt64.
I want to know what percentage is A compared to B. Namely I want to compute (A / B) * 100. This percentage must be a float64 value.
My fear is that result := float64(A) / float64(B) * 100 could potentially make a rounding error, when converting A and B to float64, then again when dividing them.
How legitimate is this fear ? What is your experience, how much of a precision loss should I be expecting ?
You said float, you will have rounding errors. No way around that.
And if B is larger than A, then using integer division will always result in 0. This is a huge error rate and can be considered a very huge rounding error.
So, by doing float conversion early, you probably reduce the rounding error…
I agree with NobbZ but want to add that you might be able to mitigate some of the rounding issues by using the math/big package’s big.NewRat or (*big.Rat).SetFrac64 and then calling (*big.Rat).Float64 which I suspect will be measurably slower than converting a and b to float64s up front but might get you more accurate results? Not sure though. When in doubt, test and measure it!
It depends if you accept some precision loss or if you want the exact result with as many decimal digits as possible.
If you accept loss in precision, you can pre-divide A and B by 4096. This correspond to 12 bits which correspond to the sign and exponent bits of the float64 number. This division ensure that the rounding is the same on A and B. The division by 4096 is canceled out with A/B.
You wont get the exact result, but the precision on the final result may be good enough for your need.
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.