From: Jim Surname <jlott@...>
Date: 2008-01-26T06:32:09+09:00
Subject: Re: Too many open files (Errno::EMFILE) on Windows

> What you're doing here is pushing the file _descriptor_ onto the
> 'files' array, not the file name, and you're never closing the
> descriptor. Simply increasing the descriptor limit isn't going to cut
> it.

Perhaps I was unclear as to what the problem is. The example code I gave 
you just points out a problem with the C run-Time API on Windows which 
is used by Ruby for handling files. The example code demonstrates how 
this limit affects Ruby programs. Your responses focus just on my 
example code, completely ignoring the rest of my message.

My hope was that someone could help clarify where the actual problem is 
and either suggest how I might work around this problem or how I might 
patch Ruby to fix it. I know for damn sure that MS will not fix the 
problem in their C run-time; otherwise, why would anyone use their 
proprietary APIs? So the only way to fix it is by patching Ruby.

In my Ruby installation folder there is a file: lib/ruby/1.8/cgi.rb
In that file is a function: read_multipart
In read_multipart there is a block of code that looks like this:
---
      loop do
        head = nil
        if 10240 < content_length
          require "tempfile"
          body = Tempfile.new("CGI")
        else
          begin
            require "stringio"
            body = StringIO.new
          rescue LoadError
            require "tempfile"
            body = Tempfile.new("CGI")
          end
        end
---

In my Rails application, I have a multipart form with about ten billion 
fields, including some file upload fields. When this form is submitted, 
the POST data gets passed to this read_multipart method in some way. The 
read_multipart method seems to be creating a Tempfile for every field 
passed from the form. When the limit for file descriptors is reached, 
the Ruby process deadlocks and I have to kill and restart it.

My temporary "fix" was to increase the limit to it's maximum which, on 
Windows, requires a recompile of Ruby. I already know that this will not 
be sufficient, otherwise I would not be posting here.

There may be a better way to write the read_multipart method so that it 
doesn't need so many Tempfiles. That would probably be easier than 
updating Ruby's File class to use something other than the standard C 
API on Windows.
-- 
Posted via http://www.ruby-forum.com/.