[ruby-core:35572] Re: [Ruby 1.9 - Feature #4531][Open] [PATCH 0/7] use poll() instead of select() in certain cases

KOSAKI Motohiro <[e-mail]> wrote:
> > ref: [ruby-core:35527]
> >
> > This adds a new C API function with the following prototype:
> >
> >   rb_io_poll_fd(int fd, short events, int timeout);
> >
> > It is emulated using select() for platforms that we do not support
> > poll() for.  It is much easier to use than rb_thread_select() and
> > rb_thread_fd_select() for the common case in C extensions[1].
> >
> > For Linux (and eventually any other platforms where poll() works for all
> > select()-able files), we actually implement rb_io_poll_fd() using the
> > poll() system call which means it is faster for high numbered file
> > descriptors and does not put malloc pressure on the garbage collector.
> 
> Meta review comment. All performance patches should have mesured
> performance number.

Based on a contrived test case:
  http://redmine.ruby-lang.org/attachments/1562/io_select_using_poll_test.rb
I get the following results:

---- before NOFILE=1024 ---
max fd: 1024 (results not apparent with <= 1024 max fd)
last IO: #<IO:fd 1022>
  2.050000   1.440000   3.490000 (  3.476264)

GC.count: 386

---- after NOFILE=1024 ---
max fd: 1024 (results not apparent with <= 1024 max fd)
last IO: #<IO:fd 1022>
  2.430000   0.320000   2.750000 (  2.734643)

GC.count: 343

---- before NOFILE=16384 ---
max fd: 16384 (results not apparent with <= 1024 max fd)
last IO: #<IO:fd 16382>
  3.610000   5.680000   9.290000 (  9.266017)

GC.count: 643

---- after NOFILE=16384 ---
max fd: 16384 (results not apparent with <= 1024 max fd)
last IO: #<IO:fd 16382>
  2.970000   0.450000   3.420000 (  3.415426)

GC.count: 394

I don't have real world results/test case but I think small bits
like this count for the future.

> And, if you really want to care C10K scalability issue, why don't you use
> epoll?

I assume you mean Ruby programming in general and not making MRI itself
use epoll().

I find synchronous programming easier especially when dealing with
existing libraries (I use Ruby to make programming life easier :)

For me, thousands of native threads (NPTL + 64-bit) often makes more
sense than epoll() + non-blocking I/O.

Concurrent connections isn't always an issue, either, but also sometimes
have many open file handles to support other tasks[1].


I mainly think select() is a horrible interface and having extension
authors to deal with HAVE_RB_FD_INIT and remembering rb_fd_term() is
dangerous.  Small bits of pressure from Rubyists can hopefully steer
other OSes towards better poll() support.

Thank you for your time!

[1] DB connection pool, large memcached pool, regular files,
    various message queues, etc...

-- 
Eric Wong