[ruby-core:69226] [Patch] Wrong behaviour on GetBNPtr

Hello,

Investigating the issue with https://bugs.ruby-lang.org/issues/10268, I found that in several places throughout ossl code the return value for GetBNPtr() is blindly passed to libcrypto. The problem is GetBNPtr() considers nil to be a valid BN object, returning a null pointer instead of raising an exception and ossl code does not handle null pointers to BN. The patch I’m sending makes a nil object no longer be considered a valid BN object.

I am currently investigating some other OpenSSL related crashes in the issue tracker that seems to branch from this condition.

diff --git a/ext/openssl/ossl_bn.c b/ext/openssl/ossl_bn.c
index c503708..191b100 100644
--- a/ext/openssl/ossl_bn.c
+++ b/ext/openssl/ossl_bn.c
@@ -96,6 +96,8 @@ GetBNPtr(VALUE obj)
        }
        WrapBN(cBN, obj, bn); /* Handle potencial mem leaks */
        break;
+    case T_NIL:
+       break;
     default:
        ossl_raise(rb_eTypeError, "Cannot convert into OpenSSL::BN");
     }

Danilo