From: Robert Klemme <shortcutter@...>
Date: 2012-01-24T18:32:41+09:00
Subject: Re: Microrant on Ruy's Math Skills

On Tue, Jan 24, 2012 at 1:57 AM, Chad Perrin <code@apotheon.net> wrote:
> On Tue, Jan 24, 2012 at 08:19:06AM +0900, Ryan Davis wrote:
>>
>> Except for that whole "parse time is different from run time" part you
>> seem to be blithely ignoring. If you've already parsed float literals,
>> then they're floats and are already lossy.
>
> Are you telling me that 1.1 is automatically lossy, regardless of how you
> got there?

1.1 is a decimal floating point literal.  This is translated into a
floating point number at some point in time (likely parse time).  From
there on there is only a floating point number which has a binary
representation.  Since certain decimal values cannot be exactly
represented as binary numbers there is loss the very moment the
translation occurs.

>  I guess I need to go back and refresh my understanding of the
> math, because I thought a literal decimal number was fine but one
> achieved by arithmetic was likely to contain subtle errors due to the way
> the binary math is handled.

There is imprecision in the translation and further numeric effects
which affect precision later.  The resulting imprecision (compared to
what pure math or symbolic calculations would yield) depends on

- translation of decimal literals to binary representation
- limits of the binary representation
- nature of the operations
- order of the operations
- translation back to decimal representation

Given all this it's rather surprising that output results are so close
to what one would expect. :-)  You can study the details at

http://en.wikipedia.org/wiki/IEEE_754-2008
and for example for one format:
http://en.wikipedia.org/wiki/Double_precision_floating-point_format

Kind regards

robert


-- 
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/