From: Roger Pack <rogerdpack2@...>
Date: 2010-07-03T07:38:03+09:00
Subject: Re: ANN: zozo 1.0.0 Released

> I appreciate the work you've done here, but I'd also be delighted to hear some comments or patches to help improve RubyGems' memory footprint. Did you know we're up on GitHub now? If you notice any particularly stupid/wasteful memory stuff in RG I'd love to hear about it.

I do have some thoughts on that, as I've been experimenting lately
with speeding up rubygems.

The first thing that comes to mind is that currently rubygems always
loads *full* rubygems when all you typically need is its require
capabilities:

ex:

>> $LOADED_FEATURES.grep /rubyg/
=> ["c:/Ruby19/lib/ruby/site_ruby/1.9.1/rubygems/defaults.rb",
"c:/Ruby19/lib/ruby/site_ruby/1.9.1/rubygems/exceptions.rb",
"c:/Ruby19/lib/ruby/site_ruby/1.9.1/rubygems/version.rb",
"c:/Ruby19/lib/ruby/site_ruby/1.9.1/rubygems/requirement.rb",
"c:/Ruby19/lib/ruby/site_ruby/1.9.1/rubygems/dependency.rb",
"c:/Ruby19/lib/ruby/site_ruby/1.9.1/rubygems/gem_path_searcher.rb",
"c:/Ruby19/lib/ruby/site_ruby/1.9.1/rubygems/user_interaction.rb",
"c:/Ruby19/lib/ruby/site_ruby/1.9.1/rubygems/platform.rb",
"c:/Ruby19/lib/ruby/site_ruby/1.9.1/rubygems/specification.rb",
"c:/Ruby19/lib/ruby/site_ruby/1.9.1/rubygems/source_index.rb",
"c:/Ruby19/lib/ruby/site_ruby/1.9.1/rubygems/builder.rb",
"c:/Ruby19/lib/ruby/site_ruby/1.9.1/rubygems/config_file.rb",
"c:/Ruby19/lib/ruby/site_ruby/1.9.1/rubygems/command.rb",
"c:/Ruby19/lib/ruby/site_ruby/1.9.1/rubygems/command_manager.rb",
"c:/Ruby19/lib/ruby/gems/1.9.1/gems/gemcutter-0.5.0/lib/rubygems/commands/migrate.rb",
"c:/Ruby19/lib/ruby/gems/1.9.1/gems/gemcutter-0.5.0/lib/rubygems/commands/tumble.rb",
"c:/Ruby19/lib/ruby/site_ruby/1.9.1/rubygems/local_remote_options.rb",
"c:/Ruby19/lib/ruby/site_ruby/1.9.1/rubygems/remote_fetcher.rb",
"c:/Ruby19/lib/ruby/site_ruby/1.9.1/rubygems/gemcutter_utilities.rb",
"c:/Ruby19/lib/ruby/gems/1.9.1/gems/gemcutter-0.5.0/lib/rubygems/commands/webhook.rb",
"c:/Ruby19/lib/ruby/site_ruby/1.9.1/rubygems/version_option.rb",
"c:/Ruby19/lib/ruby/gems/1.9.1/gems/gemcutter-0.5.0/lib/rubygems/commands/yank.rb",
"c:/Ruby19/lib/ruby/gems/1.9.1/gems/specific_install-0.2.3/lib/rubygems/commands/specific_install_command.rb",
"c:/Ruby19/lib/ruby/site_ruby/1.9.1/rubygems.rb"]

There's lots of unused stuff in there.

Suggestion: load a skeleton of files by default, until needed.

Beyond that, rubygems currently takes a  relatively long time to load
all the spec files (especially rdoc-data gem), when in reality all it
needs to know is the gem version and which requireable files are in
the require_paths of each gem.  It can lazy load full specs when
necessary.

Suggestion: keep a cache file with all vital information in the root
of each gem path, ex:

.../gems/1.9.1/.rubygems_cache

A further optimization is to use marshal for the cache, to avoid
having to load YAML by default, etc.

I've been experimenting with with with faster_rubygems [1] and it
speeds up startup for jruby from 1.6s to 0.3s.

This avoids mounds of file stats and require's, and actually makes
ruby fast to start on windows.

I'd be happy to integrate something like this into normal rubygems, if
there's any interest...

Thanks!

-roger

[1] http://github.com/rdp/faster_rubygems