From: Hiroshi Nakamura <nakahiro@...>
Date: 2011-07-26T15:25:18+09:00
Subject: [ruby-core:38513] [Ruby 1.9 - Bug #2768] SEGV when using OpenSSL::Cipher (AES)  in certain way


Issue #2768 has been updated by Hiroshi Nakamura.


This patch should fix the SEGV. Still giving much thought whether it's safe to apply or not...

Index: ext/openssl/ossl_cipher.c
===================================================================
--- ext/openssl/ossl_cipher.c	(revision 32672)
+++ ext/openssl/ossl_cipher.c	(working copy)
@@ -102,6 +102,7 @@
     EVP_CIPHER_CTX *ctx;
     const EVP_CIPHER *cipher;
     char *name;
+    unsigned char key[EVP_MAX_KEY_LENGTH];
 
     name = StringValuePtr(str);
     GetCipherInit(self, ctx);
@@ -113,7 +114,14 @@
     if (!(cipher = EVP_get_cipherbyname(name))) {
 	ossl_raise(rb_eRuntimeError, "unsupported cipher algorithm (%s)", name);
     }
-    if (EVP_CipherInit_ex(ctx, cipher, NULL, NULL, NULL, -1) != 1)
+    /*
+     * The EVP which has EVP_CIPH_RAND_KEY flag (such as DES3) allows
+     * uninitialized key, but other EVPs (such as AES) does not allow it.
+     * Calling EVP_CipherUpdate() without initializing key causes SEGV so we
+     * set the data filled with "\0" as the key by default.
+     */
+    memset(key, 0, EVP_MAX_KEY_LENGTH);
+    if (EVP_CipherInit_ex(ctx, cipher, NULL, key, NULL, -1) != 1)
 	ossl_raise(eCipherError, NULL);
 
     return self;

----------------------------------------
Bug #2768: SEGV when using OpenSSL::Cipher (AES)  in certain way
http://redmine.ruby-lang.org/issues/2768

Author: Julian W��lde
Status: Assigned
Priority: Normal
Assignee: Hiroshi Nakamura
Category: ext
Target version: 1.9.3
ruby -v: ruby 1.9.1p378 (2010-01-10 revision 26273) [x86_64-linux] 


=begin
 this command gives me an SEGV on my boxes:
 
 ruby -e 'require "openssl";OpenSSL::Cipher::AES128.new("ECB").update "testtesttesttest"'
 
 while this command does not:
 
 ruby -e 'require "openssl";OpenSSL::Cipher::AES128.new("ECB").update "testtesttesttes"'
 
 A friend of mine reproduced it on his mac(1.8.6). It also worked on a server where ruby 1.8.7 (2008-08-11 patchlevel 72) [x86_64-linux] is running.
 
 It works for any kind of Blockciphermode, but not for other ciphers (e.g. BF)
=end



-- 
http://redmine.ruby-lang.org