[ruby-talk:23882] RubyGems Discussion

Wow, there has been a lot of discussion related to RubyGems over the
past few days.  I'd like to address all the ideas and suggestions
brought up.

1) RubyGems files as valid Ruby programs.

   This is an interesting idea and one that I never really considered.
But is has a lot of drawbacks and really only one main benefit.  I'll
start with the benefit: gems that are directly executable by the ruby
interpreter (without a need to modify the interpreter.)  But I've
mentioned several alternatives to achieve this functionality without
having to make the entire gem file a valid Ruby program.  Other ideas
that were mentioned relating to this include having custom install
behavior.  I think that in general this should be avoided for pure Ruby
libraries and I would consider it to be bad design if this was needed.
But if this really is needed another method could be used (such as a
special install script in the gem that RubyGems will automatically run
at installation time.)  C libraries are a whole different issue that
I've yet to spend much time thinking about (but I will eventually.)

The drawbacks to this include:
   - a format that is hard to read by the RubyGems system.  Most of the
time gems will be libraries that will be accessible from 'require' like
any other library.  But a Ruby format would require some strange 'eval'
tricks to read them in (and wrapper classes like Rich Kilmer suggested)
or a finished RubyInRuby parser which is complete overkill for this
purpose.  Also imagine trying to find a single file in a gem on a system
with 100s of gems.  You would have to load and eval each one until you
found the one with that file.  This could be insanely slow.

On that note this format isn't ideal for my current design for RubyGems.
In the current design gem metadata and the file list is cached but the
actual file contents are only read when they are needed (to save
memory.)  There is no need for an entire Ruby file to sit around in
memory in a String when it is normally only evaluated once.  So
generally loading files from RubyGems is as fast as Ruby loading files
from disk (one seek and read) and also doesn't clog up memory.

   - a format that is hard to create automatically.  Having to write a
code generator and use base64 encoding and all that just to create a
file that could just as well be a simple binary or text format is not
worth it.  Or it could be worse if people having to code the gems
themselves.  I'm creating RubyGems to make the Ruby developer's life
easier, not harder.

   - an easily modifiable format.  This probably isn't the best argument
since it is sort of based on the "security through obscurity" concept,
but it seems like if the gems are in Ruby then people might be tempted
to change their metadata and what not.  This is something that should be
avoided.  And the system will probably have checks anyways to avoid this
but still having a binary format might be better.

I haven't had a lot of time to really think about this and some of the
code snippets were neat, but in general I don't think I like it.

2) Different dependency types

   This is something I'm certainly open to, especially since I haven't
coded dependency management yet.  Also Debian's package system was part
of the influence in RubyGems, so I'm open to seeing what they have done.

3) Version as a class

   It already is one.  Of course it isn't used the same as it would be
if gem files were coded in Ruby, as Dave, Robert and Rich were
describing.

4) Multiple format types

   I actually don't think this is a bad thing and have already coded two
different file formats.  I did this so that new formats could be added
without breaking all the already created gems.  But the main thing with
these different file formats is that they need to support the same
semantics: 

   - quickly reading the gem metadata without looking at the rest of the
file.
   - quickly reading the file list without looking at the rest of the
file.
   - quickly reading one of the embedded files without looking at the
rest of the file.

As I expressed above, the Ruby code file format would not easily support
these various operations.  But due to the nature of the RubyGems system
you guys are free to try it out.  I plan to have properties in the
RubyGems properties file that easily allow you to specify new file
formats without having to touch the RubyGems code.

5) Gems as classes following some contract

   I actually thought about this in the very beginning when planning
RubyGems, but I decided against it since it just adds more work for the
developer.  I want the transition from tarball to gem incredibly easy.
It currently is (at RubyConf I demonstrated turning NQXML into a gem in
under 30 seconds.)  The more difficult we make it to use RubyGems the
slower it will be adopted.

Ryan Leavengood