[ruby-talk:5815] Some Q&A with Kahan on IEEE 754 (long)

[Looks like this never made it to the newsgroup; resending through 
ruby-list.]

Hi,

I thought the following post from comp.lang.python (and its first 
follow-up)
might be of interest to people doing math  stuff in Ruby.

Huaiyu Zhu <[e-mail]> wrote:
====================================================================
In the past few days I asked Prof Kahan several questions about IEEE
754.
He kindly replied to my questions promptly (but he warns he's falling
behind
40 of 100 emails a day, so I'm unlikely to ask him again).  He gave
permission to forward his comments here.  My questions are included here
to
provide context.  (And to reveal my ignorance. ;-))

Huaiyu

[Huaiyu Zhu]
> ...  As I understand, the intention of IEEE 754 is for
> every exception to raise a flag, so that the programmer can choose to 
deal
> with it, or not.  This is considered a good thing compared with aborting 
the
> computation unless an error is trapped.

[W. Kahan]
   The intention is ...
   0)  Ensure that a program's behavior is predictable without stopping
it and 
       losing control.
 
   1)  Allow programmers to deal with exceptions where most appropriate
for 
       their programs,  sometimes ...
        a)  Testing operands in advance to prevent exceptions
        b)  Testing results immediately after an operation to detect
exceptions
        c)  Testing flags some time afterwards to catch previous
exceptions.
       The ability to postpone judgment is as important for programs as
for
       anything else in life.
 
    2)  Allow programmers to ignore exceptions with a predictable risk, 
made 
        as small as we can by apt choice of default values,  of
subsequent 
        malfunction.

[Multiple choices]
> Python is a scripting language that is available on a variety of 
platforms,
> not all of which support IEEE 754.  On IEEE-enabled platforms there are
> choices of whether to enable it, and what to do with the exceptions.  I
> would like to hear your opinion on the following choices:
> 
> 1. Enable IEEE whenever possible.  Let all f.p. exception go through 
without
>    any action.  The user could test for NaN and Inf afterwards.

     NO!  This is what  Java  does,  and it is a serious mistake.

>
> 2. Enable IEEE whenever possible.  Each f.p. exception produces a Python
>    exception, which would halt the program unless trapped and handled.

     This is an option worth integrating with a debugger,  but it must
not 
     be the default.  Do you really wish to see  Python  programmers
trying
     to program their own trap handlers?  The expreience with  PL-1  and 
     BASIC  trap handlers is ominous;  they enlarge enormously the
capture-
     cross section for mistakes.

>
> 3. Enable IEEE whenever possible.  Let underflow go through, but let 
each
>    overflow generate a Python exception.

     NO!  The programmer is the only one competent to decide which
exceptions
     his program can ignore,  which to allow to pass on to detection
later, 
     which to trap if they must.

> 
> 4. Enable IEEE whenever possible.  Let a special module contain all the
>    flags, without changing program flow.  Programers could choose to 
examine
>    these flags or choose to ignore them.

     Yes; and provide quick and convenient ways to save-and-restore and
     merge exception flags.  Other facilities should be provided too,
but I
     cannot spare the time just now to discuss them in detail.
 
     An example worth study is the exception-handling provided by Apples
old
     SANE for the 680x0-based Macintoshes.  It was published about a
decade
     ago by Addison-Wesley.

> 
> 5. Do not enable IEEE until there is universal support, or at least 
until it
>    can be faked on those other platforms.

     Upon what hardware that does not support IEEE 754 do you expect
Python
     to run?  There is substantial support for IEEE 754 in C-9X
compilers
     too; and Borland's C for the Pentiums provides some too.  Of
course,
     you have to provide Software flags (declared variables) for
exceptions
     rather than give programmers direct access to the hardware flags
and
     control register.

[Huaiyu Zhu]
> The current problem with IEEE 754 support in Python is that, at least as 
it
> is perceived, there is no general way to turn it on on every 754-enabled
> platform and expect to obtain the same kind of signalling, etc.

[W. Kahan]
  I don't understand why you cannot package the interface to the  IEEE
754 
  flags and implement this relatively small package as required 
(different 
  for different compilers/hardware platforms)  with your distribution of 
  Python.  Of course,  the work involved in producing the first package
or 
  two will be appreciable;  but it gets easier after that  PROVIDED  you 
  choose compilers that have the necessary hooks.

[Huaiyu Zhu]
> Another
> problem is that the leading authorities of the language plan to unify 
the
> behavior in such a way as to raise exceptions for Overflow, 
Divide-by-Zero,
> Invalid Operation, while letting through Underflow and Inexact 
Operations.
> They consider this as an interim step toward full IEEE 754 support. 
> Since Python exceptions have to be caught, this would make life 
extremely
> difficult for vectorized computations during this period.

[W. Kahan]
  "Interim steps"  have a tendency to become permanent in our industry, 
where
  "Compatibility"  is the way the sins of the fathers are inflicted upon
the 
  third and fourth generations ... . 
W. K.
=====================================================================

Kevin Jacobs wrote:
=====================================================================
Huaiyu Zhu <[e-mail]> wrote:
> In the past few days I asked Prof Kahan several questions about IEEE 
754.

I'm glad to see that Dr. Kahan's opinions are in line with what
_everyone_
here wants.  I am drafting a PEP and reference implementation that
embodies
these ideas and goals:

  1) Support for setting and testing special IEEE 754 floating-point
values
     (i.e. NaNS, NaNQ, Inf, denormals).

  2) Flexible delivery and recovery from floating point traps:

        These will be deliverable as Python exceptions, handled by a
        user-specified Python function or queried by sticky status bits. 
        All options will be setable, maskable, and mergeable at runtime.

  3) A reasonable default set of IEEE 754 traps enabled.

        Current plan: Invalid OP, Divide-by-zero, and overflow.

  4) Rounding mode and precision control if supported by the platform.

  5) Thread specific handling of traps with creation-time inheritance of
     settings.

  6) Integration with Numeric Python, MatPy, and other common numerical
     extensions via sensible additions to the Python and C APIs.
 
  6) A practical software-only emulation of some of these features for
     non-IEEE754 systems that doesn't kill performance.

  7) Available in the Python 2.1 time-frame, assuming consensus can be
     reached, problems solved, and damsels saved.

Of the many issues to be resolved, here are a few things to consider:

  1) Handling of signaling NaNs.  This can get expensive since it
involves a
     test every time a floatobject is used or requires mucking about
with
     signal handlers.

  2) Speaking of signal handlers, Tim Peters stated that returns are not
     allowed from sIGFPE handlers.  I have yet to see this restriction
in
     any platform documentation.  What is the deal?

  3) Thread support -- Python threading will have to know about
     per-thread floating point data-structures.  There are also
implications
     for uthreads.

  4) I am going to need help implementing support for many of the
commonly
     used Python platforms.  I have access to Windows 95/98/NT/2000,
     Compaq/Digital Unix, AIX (Power3), IRIX, Linux (i386+glibc2.1+),
and
     Solaris.  Edward Jason Riedy has offered to help with AIX, Linux,
     Solaris and Unicos/mk.  We still need volunteers for the following:

        Windows
        OS/2
        BeOS
        VxWorks
        OpenVMS
        QNX
 
        Others?  Speak now!

Just a few comments:

> [Huaiyu Zhu]
>> 2. Enable IEEE whenever possible.  Each f.p. exception produces a 
Python
>>    exception, which would halt the program unless trapped and handled.

[Dr. Kahan]
>      This is an option worth integrating with a debugger,  but it must 
not
>      be the default.  Do you really wish to see Python programmers 
trying
>      to program their own trap handlers?

Actually, I do want to see Python programmers programming their own trap
handlers... in Python!  It'll be slow, but very much in line with the
dynamic nature of Python.  ;)


> [Huaiyu Zhu]
>> 4. Enable IEEE whenever possible.  Let a special module contain all the
>>    flags, without changing program flow.  Programers could choose to
>>    examine these flags or choose to ignore them.

[Dr. Kahan]
>      Yes; and provide quick and convenient ways to save-and-restore and
>      merge exception flags.  Other facilities should be provided too, 
but
>      I cannot spare the time just now to discuss them in detail.

I think the problem here is in the definition of "Enable IEEE".  Your
meaning seems to be "Enable IEEE" == "Enable all/some IEEE 754 traps".
My
meaning, and I believe the standard one is "use features available in
conforming IEEE 754 implementations." This is, by necessity, a superset
of
the generic C floating point support (with a few notable, but minor
exceptions).

> [Huaiyu Zhu]
>> The current problem with IEEE 754 support in Python is that, at least 
as
>> it is perceived, there is no general way to turn it on on every
>> 754-enabled platform and expect to obtain the same kind of signalling,
>> etc.

Yes and no.  No, there is no general way to support IEEE 754 in Python. 
Its
still a bloody mess in terms of APIs.  However, I think there is much we
can
do to provide as much as possible, for as many platforms as possible, in
the
near future.

> [Huaiyu Zhu]
>> Another problem is that the leading authorities of the language plan to
>> unify the behavior in such a way as to raise exceptions for Overflow,
>> Divide-by-Zero, Invalid Operation, while letting through Underflow and
>> Inexact Operations. They consider this as an interim step toward full
>> IEEE 754 support.  Since Python exceptions have to be caught, this 
would
>> make life extremely difficult for vectorized computations during this
>> period.

In the short term, i.e. Python 2.0, yes.  Its simply not feasible to do
all
that is necessary (not the least of which write a PEP, implement it and
test
for n platforms) before Python 2.0 goes out.  While important, this
isn't
important enough to even consider delaying 2.0.

I look forward to working with you to address these issues (then I can
get
back to working on statistical extensions to Python -- my real goal for
all
this silliness!)
=====================================================================

Conrad Schneiker
(This note is unofficial and subject to improvement without notice.)