Re: Stateful I/O interface

On 2/22/07, Tony Arcieri <[e-mail]> wrote:
>
>
> Kernel#select scales O(n) due to the need for the kernel to process the
> list of file descriptors to monitor with each system call.  With stateful
> interfaces like epoll and kqueue, the scalability is O(1) as the kernel does
> its event reporting on-the-fly and returns all events to the queue where
> they can be retrieved via a system call.
>
> Integrating ruby's event loop with a library that uses a
> > different mechanism requires that the library be able to use ruby's I/O
> > multiplexing system, but may not be possible or desirable, especially if
> > ruby is embedded in an application.
>
>
> Unless I'm misunderstanding the way rb_thread_select and
> rb_thread_wait_fd_rw are implemented (both tie into a single underlying
> do_select call), a different mechanism could operate alongside select calls
> provided it only blocked for the scheduler quantum.  This way calls to
> select and a stateful event monitoring interface could alternate within
> Ruby's event loop, allowing both to run (effectively) simultaneously.
>
> Implementing the rb_thread_select and rb_thread_wait_fd_rw calls with a
> stateful interface like epoll or kqueue would probably not be beneficial
> because the set of monitored file descriptors must be processed with each
> call anyway (and are subject to change).  I'd suggest these interfaces
> remain select() based for the time being (as they use select's semantics
> anyway) although they could potentially be optimized with poll(), although
> benchmarking would be required to determine if a poll()-based implementation
> were actually advantageous.
>
> epoll and kqueue both provide extremely powerful interfaces which could
> serve as the central underlying system call behind an event loop, however in
> the case of Ruby this would require a dramatic rearchitecting of the way the
> event loop functions.  That's why I'm suggesting something dramatically
> simpler: providing a simple bridge between these system calls and Ruby which
> can run within the existing event system.
>



Tony, the way I've implemented epoll in a ruby context is to constantly
interrupt the loop and allow ruby's thread scheduler to run. This is
basically the same way ruby interacts with select. If you do this at least
100 times a second, you're not hurting ruby any, because its thread
scheduling quantum is 10 mills (on Linux, anyway, which is the only platform
that has epoll). If there are no other green threads in the process, this
doesn't add noticeable overhead. If there are threads, then ruby is so much
slower anyway that it doesn't really matter.

I've done kqueue before but never with Ruby. But it's conceptually similar
to epoll so I'd guess the same comments apply.

I once spent a few days trying to rewrite EventMachine to use IOCP. What a
pain in the ass. (And I've done a *lot* of IOCP programming in pure C++
projects.) I gave up on it because Windows isn't important to any of my own
work.