View Full Version : [Article] Rounding errors
ZaiRoN
September 7th, 2006, 14:15
David Goldberg: What every computer scientist should know about floating-point arithmetic
Available here: http://docs.sun.com/source/806-3568/ncg_goldberg.html
Thx to Que for the link.
SiGiNT
September 8th, 2006, 09:22
I was shocked when I accidentally encountered this about 15 years ago, while writing an estimating program for our process in GW Basic it was necessary to use a for - to - step, operation - try this if you still have an old copy of line based basic -
10 for x = 1 to 20 step .1
20 if x = 10 then print "I'm Here!"; x
30 next x
You'll never get the "I'm here" message, somewhere shortly after about 2 or 3 you start accumulating decimal places and you never exactly hit 10.00000 - appalling, considering any TI99 or Commodore had absolutely no problem with the code.
SiGiNT
Silver
September 8th, 2006, 12:53
Ah, floating point accuracy, the bane of 3d coders everywhere...
Powered by vBulletin® Version 4.2.2 Copyright © 2018 vBulletin Solutions, Inc. All rights reserved.