From: charlez.leu@...
Date: 2015-12-10T00:51:11+00:00
Subject: [ruby-core:72017] [Ruby trunk - Bug #11799] Object allocation during garbage collection phase terminates the Ruby process

Issue #11799 has been updated by Charles Leu.


Koichi Sasada wrote:
> https://github.com/rdp/google_hash/blob/master/ext/template/google_hash.cpp.erb#L170
> 
> I'm not sure the implementation details, but
> 
> >  for(<%= type %>_hash_map< <%= key_type %>, <%= value_type %> <%= extra_hash_params %> >::iterator it = incoming->hash_map->begin(); it != incoming->hash_map->end(); ++it) {
> 
> this iteration calls #== method.


**Thanks for the prompt attention and feedback.  It's greatly appreciated.**

google_hash will be removed (unfortunately it was being relied upon heavily), and I'll re-test.
If other gems (e.g. yajl-ruby) are also problematic I'll remove their usage as well.

----------------------------------------
Bug #11799: Object allocation during garbage collection phase terminates the Ruby process
https://bugs.ruby-lang.org/issues/11799#change-55427

* Author: Charles Leu
* Status: Third Party's Issue
* Priority: Normal
* Assignee: 
* ruby -v: ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-linux]
* Backport: 2.0.0: UNKNOWN, 2.1: UNKNOWN, 2.2: UNKNOWN
----------------------------------------
Resurrection of Bug #11549 that was apparently inadvertently deleted.
Also note the similarity to Bug #10868.

**Summary:** Multi-Threaded Ruby apps are often problematic; especially so when utilizing thread pools, and scheduling work to worker threads.

* While the problem isn't easily recreated, it can be reliably recreated given sufficient time and conditions (for garbage collection in conjunction with sibling thread behavior).
* Initially, three trace files have been attached.  Each shows the problem in different conditions, and using different client infrastructure.
* If this is a problem due to usage of certain gems that aren't 'well-behaved', it would be good to know which ones to avoid.

RE: attached file ruby_2.2.3_obj_alloc_gc_bug.txt

> Following is the section of sap_consumer_control.rb that is presented by the Ruby interpreter as being the current execution context when the problem occurs.
> 
> 408: loop do
> 409: @worker_threads.schedule(@work_queue.pop, &@consumer)
> 410: @sap_packets_consumed += 1
> 411: end
> 
> Notes:
> * @work_queue is a Ruby Queue (allocated within the main thread) into which a producer thread places work requests.
> * @worker_threads is a thread pool (allocated within the main thread).
> * @worker_threads schedule method simply puts a work request into the thread pool's internal work queue. One of the worker threads within the thread pool will consume/effect the work request, by executing the specified consumer Proc.
> * The main program thread simply loops forever scheduling work to thread pool threads.
> * It appears that an object is being allocated by virtue of the @work_queue.pop

RE: attached file ruby_bug_redis_client.txt

> * The observed behavior also occurs readily when employing the Redis client gem. 
> * This is the use case in which the bug is most easily reproduced.

RE: attached file ruby_bug-gdb.txt

> * The observed behavior has also been observed when using stretcher, faraday, net-http-persistent gems.
> * Connections via stretcher to net-http-persistent are on a per thread basis (maintained within thread local storage).
> * In this particular case, the Ruby process was run under gdb, and back traces for each of the process threads are provided.
 


---Files--------------------------------
ruby_bug_redis_client.txt (53.3 KB)
ruby_2.2.3_obj_alloc_gc_bug.txt (49.1 KB)
ruby_bug-gdb.txt (151 KB)


-- 
https://bugs.ruby-lang.org/