floating point bug?? 
Author Message
 floating point bug??

I wrote some code to compare decimal numbers after some calculations, but upon
comparing, they don't match!
 eg. <cfoutput>#(123.45 - 23.4 eq 100.05)#</cfoutput>

 the result is a resounding NO!

 a check reveals this (123.45 - 23.4 - 100.05) is actually equals to  
1.42108547152E-014 instead of 0

 what 's going on??



Fri, 16 Feb 2007 16:18:42 GMT
 floating point bug??

Well, floating point functions on all computers are an approximation, but your example is not even close. Can you post more of your code to give more context to the issue?


Fri, 16 Feb 2007 20:17:36 GMT
 floating point bug??

Quote:
>  eg. <cfoutput>#(123.45 - 23.4 eq 100.05)#</cfoutput>

As someone else posted, FP numbers on computers are only ever
appromixations.  They are derived sums of inverses for binary numbers,
thus:

0.75 = 1/2 + 1/4

0.33 = 1/4 + 1/16 + 1/64 + 1/256 [etc]

So you're not going to get accurate answers with decimal arithmetic.

Also note that FP numbers are only stored with a specific degree of
accuracy, for example only storing the first eight digits of the number,
and a power of ten to muliply it by.

Adam



Sat, 17 Feb 2007 04:33:13 GMT
 floating point bug??
Actually,
 #Numberformat(123.45 - 23.4 - 100.05,"9.99999999999999999999")#
 shows the value as
 0.00000000000001421085
 Not zero but very small
 1.42108547152E-014 is scientific notation for the same very small number


Sat, 17 Feb 2007 04:50:21 GMT
 floating point bug??
Try <cfoutput>#(Val(123.45 - 23.4) eq Val(100.05))#</cfoutput> and see what
you get


Quote:
> I wrote some code to compare decimal numbers after some calculations, but
upon
> comparing, they don't match!
>  eg. <cfoutput>#(123.45 - 23.4 eq 100.05)#</cfoutput>

>  the result is a resounding NO!

>  a check reveals this (123.45 - 23.4 - 100.05) is actually equals to
> 1.42108547152E-014 instead of 0

>  what 's going on??



Sat, 17 Feb 2007 09:54:32 GMT
 
 [ 5 post ] 

 Relevant Pages 

1. floating point bug report( 6.02 )

2. floating-point number bug?

3. How do I round floating-point numbers?

4. floating point numbers

5. round up floating point to hundredths

6. floating point?

7. Floating Point truncation?

8. Frame rate with a floating point number

9. Floating point numbers

10. Floating point

11. Floating point parameter problem

12. Floating point number query


 
Powered by phpBB® Forum Software