Re: Microrant on Ruy's Math Skills

On Sat, Jan 28, 2012 at 05:26:01AM +0900, Gary Wright wrote:
> On Jan 27, 2012, at 1:39 PM, Chad Perrin wrote:
> >
> > If you think of 1.1 as notation for a much more complex floating point
> > number, which is not the same as 1.1, that doesn't mean the abstraction
> > doesn't exist: it means you're unravelling it in your head to accommodate
> > the implementation's divergence from decimal 1.1.  In essence, the fact
> > it looks like 1.1 (but isn't) is the abstraction itself.
> 
> I think you are right about the leakiness of the floating point internal
> representation vs. external representation.
> 
> What I do find surprising when this discussion pops up (and it pops up
> with frightening regularity on ruby-talk) is that there are so many
> programmers who are unaware of the issues surrounding floating point
> representation.  This is not a Ruby issue but is actually a very common
> situation across a huge number of programming languages.
> 
> I found <http://rosettacode.org/wiki/Literals/Floating_point> to be
> an interesting laundry list of floating point literals. Almost every
> language defaults to having decimal floating point literals.
> A couple variations:
> 
> -- ISO C99 has hexadecimal floats: 0x1.fp3  = ( (1 + 15/16) * 2^3 )
> -- PL1 has binary floats: 111.0101e7b = (111.0101 * 2^7) = (7.3125 * 2^7)
> 
> I'm sure there are still some gotcha's regarding the mapping of abstract
> hex or decimal floats into the reality of the underlying hardware
> representation.

There's always the matter of limited room for precision in memory.  The
question of what happens when you reach that limit then comes up (usually
resulting in truncation, I think).


> 
> A more common approach (but not universal) is support for fixed point
> decimal literals and arithmetic. For example, most SQL implementations
> have support for fixed point arithmetic and literals.
> 
> <http://en.wikipedia.org/wiki/Fixed-point_arithmetic#Implementations>

I'd be happy with a fixed point implementation that offers very
intuitive -- or at least succinct and expressive -- syntax.


> >
> > This is where the special comparison method
> > proposals make sense: if such a method can guarantee that it is accurate
> > up to a known, "standard" precision, it's easy to think "Floats are as
> > they appear up to precision X," and just move on with your life, because
> > it works; without them, we only have something like == as currently
> > implemented for Float, whose primary value (as far as I can see) is to
> > provide a tool for learning about the implementation of the Float type,
> > because there's no simple rule of thumb for "accuracy up to precision X".
> 
> Why the ill-will towards Float#==?  Despite the problems associated with
> floating point representation and computation I don't see how discarding
> or modifying the semantics of #== would help the situation.

It's not ill will toward Float#== (at least for me).  It's disappointment
that, when I want to do something that fits the common case, I have to
implement it myself or use something like BigDecimal (which is far from
succinct in how it is used).


> >
> > what we have is the need to implement a
> > comparison method of our own individual choosing every single time we
> > want to be able to rely on accuracy of decimal math.
> 
> Only if you insist on using floating point values as a substitute for
> real decimal values (e.g. BigDecimal or something similar). Even then
> you need to be aware of how the results of arithmetic computations
> are going to be stored.  What 'value' do you expect for this expression:
> 
> 	BigDecimal("1.0") / BigDecimal("3.0")
> 
> It can't be an exact representation of the arithmetic result within
> the context of BigDecimal.  So you can switch to Rational:
> 
> 	Rational(1) / Rational(3)

These things are much more cumbersome to use than something I might
implement myself as a comparison method for Float -- so for any cases
where the usage is nontrivial (that is, where the very verbose syntax
needs to be employed over and over and over again), they might actually
not be the best option.  The point I was make was that the situation is
far from ideal, or even just mildly inconvenient a lot of the time.


> 
> Fantastic.  You've now got 1/3 stored internally.  What are you going
> to do when you want to throw that up on a web page or export it to a
> CSV file to be imported into a spreadsheet?  Probably convert it to
> a decimal floating point value but how exact do you want to get:
> 
> "%.60f" % (Rational(1)/Rational(3)).to_f
> => "0.333333333333333314829616256247390992939472198486328125000000"
> 
> Hmm. That introduces the decimal/binary problem.  How about:
> 
> >> (Rational(1)/Rational(3)).to_d(20).to_s('f') #=> "0.33333333333333333333"
> >> (Rational(1)/Rational(3)).to_d(30).to_s('f') #=> "0.333333333333333333333333333333"
> >> (Rational(1)/Rational(3)).to_d(70).to_s('f') #=> "0.3333333333333333333333333333333333333333333333333333333333333333333333"

Can you not see how ugly and cumbersome that is?


> 
> Of course what happens when you want to compute something like square root
> with rational values?  The result can't be exactly represented as a rational
> so you are back to the representation problem:
> 
> >> Rational(2).to_d(20).sqrt(20).to_r
> => (5656854249492380195206754896838792313/4000000000000000000000000000000000000)
> 
> No magic bullets. You still have to think about what format/representation
> is appropriate for your use.

Thank you for making my point: it would be nice to have a "standard" way
of doing things where, for the majority of cases, the results are
unsurprising *without* having to consciously account for intermittently
weird results.


>  >
> > Note that a decimal "up to precision X" is also an abstraction, but at
> > least it is an abstraction that would leak far, far less often, because
> > of the case of things like rounding.  I think the only way around that,
> > given the fact there are limits to how much RAM we have available, would
> > be to store rational literals (e.g. 2/3 instead of 0.666 . . .) somewhere
> > to provide a back-up method for rounding numbers.
> 
> Sure, you can use Ruby's Rational class if you want like shown above. Still
> doesn't get rid of the problems.

Are you even reading what I'm writing?  My point is "Let's see if we can
minimize the problem for the common case, with very clear boundaries on
when things go astray, as an alternative to the way it is currently done
by default -- which is to have the abstraction leak past boundaries that
we can only really predict by doing binary math."


> >
> > Someone tell me if I'm mistaken about some part of that -- preferably
> > without invective.
> 
> I don't think you are mistaken, but I also don't have a handle on what you
> think should happen or what is missing in Ruby.  The issues surrounding numeric
> computation (representation, overflow, accuracy, precision, conversion) are
> inherent in the problem domain.  Programmers need to be aware of them and
> use appropriate tools as necessary: Fixnum, Bignum, Float, BigDecimal,
> Rational, Complex, etc.

Look back at the beginning of what I said.  It boils down to this:

One or two alternate comparison methods for Float that have known, clear,
easily reasoned boundaries between where it works and where it does not,
should be pretty easy to include in the Float implementation -- and to
make succinct so we don't have to deal with fugliness in our code at
every turn when dealing with decimal numbers.

-- 
Chad Perrin [ original content licensed OWL: http://owl.apotheon.org ]