[ruby-core:64127] [ruby-trunk - Bug #10019] segmentation fault/buffer overrun in pack.c (encodes)

Issue #10019 has been updated by Will Wood.


Well the reproducibility issue appears to be a toolset problem.  If I build with -O2 the code works fine.  I think ultimately the optimizations (O3) for gcc 4.7.2 at least with the mingw toolset are causing the issue here for windows.  I've subsequently built with the mingw 4.9.1 toolset and it works fine with O3, so I guess there's a 4.7.2 gcc bug that's in the way here.

----------------------------------------
Bug #10019: segmentation fault/buffer overrun in pack.c (encodes)
https://bugs.ruby-lang.org/issues/10019#change-48139

* Author: Will Wood
* Status: Feedback
* Priority: Normal
* Assignee: 
* Category: core
* Target version: 
* ruby -v: ruby 2.1.2p168 (2014-07-06 revision 46721) [i386-mingw32]
* Backport: 2.0.0: REQUIRED, 2.1: DONE
----------------------------------------
While working with an AWS sample I hit a segmentation fault.  The same sample works under 1.9.3.  It appeared to be coming from pack.c function encodes.  After looking at the source there's a 4K buffer allocated on the stack.  I made a minor change to base the buffer length off of the incoming buffer length with a pad and allocate it off the heap.  Anyway, after fixing this my code sample runs fine.  I'm including a patch file and the sample code.

---Files--------------------------------
pack.patch (2.74 KB)
BucketTest.rb (326 Bytes)
pack.c.patch (769 Bytes)


-- 
https://bugs.ruby-lang.org/