From: Ben Bleything <ben@...>
Date: 2010-02-17T03:31:39+09:00
Subject: Re: ruby to_i problem

On Tue, Feb 16, 2010 at 10:16 AM, Phil Cooper-king <phil@cooperking.net> wrote:
> For what ever reason the first instance gives an error, and every
> instance there after works as it should

What You Think Is Happening Is Not What Is Actually Happening :)

> ruby 1.8.7
> [irb]
> # This works the way I would think
>>> 1.0.o_i
> => 1

Yep, and every time you say 1.0.to_i you will get 1.

> # This doesn't
>>> x = -1
>>> y = -1
>>> angle = 90 * Math::PI / 180
>>> n_x = (Math::cos(angle) * x) - (Math::sin(angle) * y)
> => 1.0

irb is showing you an approximation.  It's confusing because irb is
showing you n_x to be 1.0, when in fact:

>> sprintf "%0.25f", n_x
=> "0.9999999999999998889776975"

> # this is where its werid
>>> n_x.to_i
> => 0

Yep:

>> 0.9.to_i
=> 0

> any further 1.0.to_i works

1.0.to_i and n_x.to_i (from your example) are two different things.

Search the group archives for "floating point", you're sure to find a
huge pile of other people with similar problems, and likely good
solutions.

Ben