I am working in JavaScript, but the problem is generic. Take this rounding error:

>> 0.1 * 0.2
0.020000000000000004

This StackOverflow answer provides a nice explanation. Essentially, certain decimal numbers cannot be represented as precisely in binary. This is intuitive, since 1/3 has a similar problem in base-10. Now a work around is this:

>> (0.1 * (1000*0.2)) / 1000
0.02

My question is how does this work?

I am working in JavaScript, but the problem is generic. Take this rounding error:

>> 0.1 * 0.2
0.020000000000000004

This StackOverflow answer provides a nice explanation. Essentially, certain decimal numbers cannot be represented as precisely in binary. This is intuitive, since 1/3 has a similar problem in base-10. Now a work around is this:

>> (0.1 * (1000*0.2)) / 1000
0.02

My question is how does this work?

• javascript
• math
• floating-point
• floating-point-precision

Share Improve this question Follow edited May 23, 2017 at 12:05 CommunityBot 111 silver badge asked Jun 11, 2014 at 20:54 jdsjds 8,2801313 gold badges6868 silver badges111111 bronze badges 13

• What Every Computer Scientist Should Know About Floating-Point Arithmetic – adeneo Commented Jun 11, 2014 at 20:59
• 1 Interesting question, I guess the answer must e down to where the precision in lost – Ian Commented Jun 11, 2014 at 21:01
• 1 The last one can simply be written as 0.1 * 200 / 1000, or 20 / 1000, which of course don't have the floating point issue – adeneo Commented Jun 11, 2014 at 21:01
• 1 @adeneo: That 0.1 * 200 == 20 is nontrivial and depends in a fundamental way on the binary expansion of 0.1. Compare with 0.145 * 200 < 29, for instance. This doesn't deserve to be swept under the rug with just the word 'or.' – tmyklebu Commented Jun 12, 2014 at 0:16
• 1 @adeneo: I don't believe it does. First, numbers in Javascript are always 64-bit floating-point. Second, I don't see how to reconcile the result of 0.145 * 200 with the scheme you seem to be proposing. Perhaps another look at WECSSKAFPA is in order for you. – tmyklebu Commented Jun 12, 2014 at 0:41

 |  Show 8 more ments

3 Answers 3

Sorted by: Reset to default 4

It doesn't work. What you see there is not exactly 0.02, but a number that is close enough (to 15 significant decimal digits) to look like it.

It just happens that multiplying an operand by 1000, then dividing the result by 1000, results in rounding errors that yield an apparently "correct" result.

You can see the effect for yourself in your browser's Console. Convert numbers to binary using Number.toString(2) and you'll see the difference:

Correlation does not imply causation.

It doesn't. Try 0.684 and 0.03 instead and this trick actually makes it worse. Or 0.22 and 0.99. Or a huge number of other things.

Since the numbers are constants, the expression can be evaluated before assigning the numbers to variables, so no float variables need to be used except for the one that stores the result of the expression.