From: Glenn Parker Date: 2005-04-01T01:32:10+09:00 Subject: Re: Integer/Float oddity michael.b.masi wrote: > > so the question becomes, what is different about xbasic's internal > numeric representation that allows it to give a correct answer to this > seemingly simple number manipulation? Probably nothing about the internal representation is different. The difference would have to be in xbasic's "INT()" operator, which apparently performs rounding. Dealing with floating point numbers can often produce counter-intuitive results, mostly because simple analogies based on fixed-point math fail. For example, the concept of "equality" in floating point math often clashes with common sense. You might assume that the following expression is true: (100*(1.15-1)) - 15 == 0 The computer would disagree. What you really want to test is: abs( (100*(1.15-1)) - 15 ) < TOLERANCE Where the choice of TOLERANCE depends very much on the application. -- Glenn Parker | glenn.parker-AT-comcast.net |