From: "Brian F. Feldman" <green@...>
Date: 2000-12-20T14:23:36+09:00
Subject: [ruby-talk:7737] strange border-case Numeric errors

I haven't had a good enough chance to familiarize myself with the code in 
question, so I cannot supply a fix at the moment, but I have a couple border 
case bugs.

Here are some example operations that have the wrong results:

irb(main):028:0> 0x40000000 << 2
0
irb(main):029:0> 0x40000001 << 2
4
irb(main):030:0> 0x80000000 << 2
0
irb(main):031:0> 0x80000001 << 2
4
irb(main):032:0> 0x100000001 << 2
17179869188
irb(main):033:0> 0x100000000 << 2
17179869184


The upper bit seems to be being cut off here for numbers > 0x40000000.  I 
don't know much except that because of the initial left shift for Fixnums 
and to account for the sign bit, the highest representable Fixnum will be 
0x40000000 (on i386), so I have a feeling that the border cases are not 
correctly handled and instead have off-by-one (two?) bug(s).

Please, someone with more free time than I have, see if you can track down 
the bugs :)

--
 Brian Fundakowski Feldman           \  FreeBSD: The Power to Serve!  /
 green@FreeBSD.org                    `------------------------------'