From: Eric Wong <normalperson@...>
Date: 2018-05-02T04:08:26+00:00
Subject: [ruby-core:86817] Re: [Ruby trunk Feature#14723] [WIP] sleepy GC

Koichi Sasada <ko1@atdot.net> wrote:
> OK. I assumed that this "step" API is used with "rb_gc_inprogress()". But it
> is not correct.

Right, inprogress is only a hint.  I will add a comment to that effect.
As with ppoll + read, there is always a chance of work being "stolen"
by other threads :)

> ```
> +	    if (mutex->th == th) {
> +		mutex_locked(th, self);
> +	    }
> +	    if (do_gc) {
> +		/*
> +		 * Likely no point in checking for GVL contention here
> +		 * this Mutex is already contended and we just yielded
> +		 * above.
> +		 */
> +		do_gc = rb_gc_step(th->ec);
> +	    }
> ```
> 
> it should be `else if (do_gc)`, isn't?

Yes, I will fix.

> >One problem I have now is threads in THREAD_STOPPED_FOREVER
> >state cannot continuously perform GC if some other thread
> >is constantly making garbage and never sleeping.
> 
> >       nr = 100_000
> >       th = Thread.new do
> >         File.open('/dev/urandom') do |rd|
> >           nr.times { rd.read(16384) }
> >         end
> >       end
> >
> >       # no improvement, since it enters sleep and stays there
> >       th.join
> >
> >       # instead, this works (but wastes battery if there's no garbage)
> >       true until th.join(0.01)
> 
> I'm not sure why it is a problem. Created thread do `read` and it can GC
> incrementally, or if `read` return immediately, there are no need to step
> more GC (usual GC should be enough), especially for throughput.

I suppose....   Note: read on urandom won't hit rb_wait_for_single_fd
to trigger GC(*), but it will only trigger GC via string allocation.


(*) /dev/urandom can't return with EAGAIN, only /dev/random can

> >So maybe we add heuristics for entering sleep for methods in
> >thread.c and thread_sync.c and possibly continuing to schedule
> >threads in THREAD_STOPPED_FOREVER state to enable them to
> >perform cleanup.  I don't think this is urgent, and we can
> >ignore this case for now.
> 
> "cleanup"? do GC steps? I agree on them (requirements and immediacy).

Sure.  Should I commit after adding "else" to mutex_lock?

Unsubscribe: <mailto:ruby-core-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>