From: Jeremy Evans Date: 2010-07-02T13:34:13+09:00 Subject: Re: ANN: zozo 1.0.0 Released John Barnette wrote: > On Jul 1, 2010, at 12:54 PM, Jeremy Evans wrote: >> Rubygems is a fine package distribution system, but it is not very >> efficient from a runtime memory standpoint. If your application uses >> rubygems in production, every time it starts, rubygems needs to figure >> out which packages to load. zozo makes it so that this is calculation >> is only done once, and the result is cached into a local directory. > > I appreciate the work you've done here, but I'd also be delighted to > hear some comments or patches to help improve RubyGems' memory > footprint. Did you know we're up on GitHub now? If you notice any > particularly stupid/wasteful memory stuff in RG I'd love to hear about > it. > > http://github.com/rubygems I'm sorry if I implied that rubygems is wasteful with memory. By "not very efficient" I meant that it uses a lot of memory compared to other lightweight libraries such as sequel, sinatra, and unicorn. There probably is a good reason for rubygems' memory use. If it's possible to save the ~10MB per process by doing the rubygems' calculation once and caching the result, I definitely think it's worth it, especially if 10MB is a good portion of the process's memory footprint. In terms of analyzing rubygems' memory use, I'd probably start with tmm1 and ice799's memprof: http://github.com/ice799/memprof. I haven't actually used it, but I've seen the presentations and I'm pretty sure it could tell you where rubygems is using memory. If I had to guess, it has mostly to do with how much code rubygems is loading, even without doing anything: $ ruby -e "system('ps ux | fgrep ruby')" jeremy 4207 0.0 0.1 1064 2700 p8 S+ 9:13PM 0:00.01 ruby -e syste $ ruby -rubygems -e "system('ps ux | fgrep ruby')" jeremy 5489 0.0 0.3 9200 11300 p8 S+ 9:13PM 0:00.15 ruby -rubygems -e system('ps ux | fgrep ruby') Considering how much Sequel adds: $ ruby -I lib -r sequel -e "system('ps ux | fgrep ruby')" jeremy 2488 0.0 0.3 6952 8952 pe S+ 9:25PM 0:00.12 ruby -I lib -r sequel -e system('ps ux | fgrep ruby') When you consider that rubygems' codebase is larger than Sequel's (9648 LOC for rubygems and 5548 for Sequel), it's not surprising that rubygems takes more memory. If code size is truly the reason, the only thing you can do is try to reduce the amount of code you load at once, if possible. Sequel does this by not loading adapters, connection pools, plugins, and extensions that aren't being used. Rubygems might be able to do something similar, by only loading code necessary for the purpose (i.e. only load the code for installing gems when the user uses gem install). That may cause some backwards compatibility issues, though. Jeremy -- Posted via http://www.ruby-forum.com/.