From: Edd Barrett <vext01@...>
Date: 2008-03-06T09:32:58+09:00
Subject: Too many open files on OpenBSD

Hi,

I am the maintainer of the TeXLive typesetting suite for OpenBSD. Each
year I must parse a database in order to calculate subsets of the
TeXLive system for packaging purposes.

The database format changed this year and I have thrown away my python
script and had a go at it with ruby. I am quite new to ruby, but not to
programming (previously C, C++, Java, BASIC..... )

This is the error I am stuck with:

---8<---
./tlpsrcnode.rb:19:in `initialize': Too many open files -
/home/tl/tl/Master/tlpkg/tlpsrc/wordcount.tlpsrc (Errno::EMFILE)
        from ./tlpsrcnode.rb:19:in `new'
        from ./tlpsrcnode.rb:19:in `parse'
        from ./tlpsrcnode.rb:30:in `parse'
        from ./tlpsrcnode.rb:20:in `each'
        from ./tlpsrcnode.rb:20:in `parse'
        from ./tlpsrcnode.rb:30:in `parse'
        from ./tlpsrcnode.rb:20:in `each'
        from ./tlpsrcnode.rb:20:in `parse'
         ... 3055 levels...
        from ./tlpsrcnode.rb:20:in `each'
        from ./tlpsrcnode.rb:20:in `parse'
        from ./roottlpsrcnode.rb:53:in `startParse'
        from ./rbmfsplit:54
---8<---

This appears to happen after 297 files are concurrently opened.

Immediately I dump myself into the staff login class and uncap the file
descriptor count in login.conf. Log out then back in.

---8<---
puff% ulimit -a
-t: cpu time (seconds)         unlimited
-f: file size (blocks)         unlimited
-d: data seg size (kbytes)     524288
-s: stack size (kbytes)        4096
-c: core file size (blocks)    unlimited
-m: resident set size (kbytes) 1747436
-l: locked-in-memory size (kb) 582993
-u: processes                  128
-n: file descriptors           1024
---8<---

Uncapped in this case appears to be 1024. Plenty.

The error still occurs after 297 files are open.

Is there a workaround for this or is ruby not suitable for this nature
of processing?

Thanks

Edd
-- 
Posted via http://www.ruby-forum.com/.