Re: RubyGems in Ruby HEAD

Hi all,

This reply is in the name of the Debian/Ruby Extras team.  When I write
"we", I mean this group.

On Wed, Sep 21, 2005 at 03:28:01AM +0900, Austin Ziegler wrote:
> On 9/20/05, Mauricio Fern疣dez <[e-mail]> wrote:
> > Right now, RubyGems represents a step backwards relative to Minero
> > Aoki's setup.rb in many regards as far as repackagers are concerned.
> 
> To be honest, this is one of my *least* worries. I know, Mauricio, that
> it matters greatly to you, but I find that RubyGems has solved things in
> a manner similar to stow.

But it is our most important worry!  As Debian developer this would
increase my amount of work for packaging _and_ maintaining each Ruby
library/app.  Although this trend is more related to RubyGems popularity
gain than the actual merge.

> > To state it simply, it is often more difficult to package something
> > released in .gem format than in an equivalent tarball + setup.rb. I
> > know this from my personal experience when repackaging a large number
> > of libraries and applications [2] and most importantly from what
> > several developers working on established repackaging efforts (Debian,
> > FreeBSD, PLD, Suse) have told me. 

Indeed.  Note that this is partly due to a different point of view.  For
Debian we package the libs and apps ourselves.  I hate to say it but
there is almost no need for RubyGems in our case, whereas it is very
handy and good for platforms lacking such packaging and Q&A (win32, OSX,
...).  I certainly see the point and usefulness of the system.

I am still more interested in a system to generalize _all_ Ruby lib/app
sources, like Package[2].  If RubyGems, repackagers, etc. all could use
that, that would be great.  And it makes more sense to have that in the
core IMO. 

> > This is due to RubyGems breaking
> > source compatibility with non-RubyGems system in several areas,
> > including, but not limited to the following:
> 
> > * lack of support for DATADIR
>
> [...]
> 
> > * obviously, require_gem
> 
> Which should be going away, and has always been said that it would be
> going away. The biggest problem will be the association between the gem
> name (what you install) and the libraries (what you require) and whether
> that should be maintained. I do not believe that it should.

With this gone, it would stop creating source incompatibilities.
This would result in far less patches of apps and libs for us.
Now, an app can not work because of the absense of RubyGems in Debian
(which will change soon though, but packages shouldn't depend on it,
IMHO).

The mapping between the gem and the libs (and an API to access it) might
still be useful, see also the end of the mail.

> > * in general, problems due to the new directory layout
> 
> Please expand on this, because I see nothing different between this and
> what stow does, except that RubyGems does it transparently. Indeed, with
> the latest versions of RubyGems, aside from the necessary "require
> 'rubygems'" line, there is no practical difference between what RubyGems
> does and what RPA did (excepting, of course, that RPA installed directly
> into site_ruby).

And exactly that practical difference makes it violate the FHS and
thereby our Debian policy.  Say we were planning to make a transition to
seperating arch-independant and arch-dependant libs into /usr/share and
/usr/lib dirs, with this 1-gem-1-dir approach it would be virtually
impossible.

> > Each of these items means additional work when packaging a .gem: the
> > source code must be patched before the actual repackaging work can be
> > begun. This is something repackagers wouldn't have had to do with a
> > traditional .tar.gz+setup.rb release. Since RubyGems has been proposed
> > as a standard for distribution and management of third party
> > libraries, it has become the sole release format for several projects,
> > forcing repackagers to patch an increasing amount of upstream code,
> > adding to their thankless work. 

Agreed.

> > Upstream developers aware of this problem have to write their code
> > carefully to make sure it works in both RubyGems and all other
> > systems, or to keep two lines of development and make duplicate
> > releases.  To sum up, this is a lot of work that we could live
> > without if the issues affecting RubyGems were fixed before it is in
> > more widespread use.
> 
> I personally have issues with the idea that repackagers will be
> patching my libraries.  I appreciate the work you did with RPA and the
> reviews you performed on my code, but the reality is that I'm not sure
> that repackagers *should* patch incoming code unless it is clear that
> the project has been abandoned and it is a security patch. 

I have issues with upstream adding code that has to do with packaging.
As Debian/Ruby maintainers, we have agreed that packaging is and
should always be _orthogonal_ to upstream software work.

> > I believe that "the Ruby standard for publishing and managing third
> > party libraries" should not make things any harder to package, for
> > there are legitimate reasons to prefer existent tools (rpm, dpkg,
> > etc.) to RubyGems, [...]
> 
> I disagree, but I'm obviously going to be in a minority here. I think
> that the situation on Debian demonstrates that, sometimes, the
> repackagers do far more harm than good. 

Thanks!

> I'm glad that it's better, but it never should have been the mess that
> it was.

If this is about the stdlib being split up.  That was a choice, and I
can understand the maintainer's choice, it was a more logical thing for
Ruby1.6 than 1.8, stdlib being much smaller. Some people are just
dependency-nit-pickers.  

> > To make things clear, I'll repeat once more that I'm not opposed to a
> > Ruby standard being adopted for upstream releases, or to RubyGems
> > becoming such a standard after all the above issues have been dealt
> > with. But I believe the current RubyGems implementation shouldn't be
> > considered the sanctioned standard before that happens.
> 
> I think that you need to clarify the issues. Everything you've stated
> here has been, IMO, quite vague. It may be useful to highlight the
> issues with specific cases *and provide suggested solutions*. 

The issues may sound vague because they are quite general (source
incompatibility, being able to run stuff without gems, FHS compliance)
and most count for all gems or the entire system.

> Those solutions may involve changes that can only happen when RubyGems
> is incorporated in Ruby, but let's be realistic here.  If the RubyGems
> developers aren't involved in repackaging efforts, those issues are
> going to end up being low on their priority efforts unless someone
> comes to them with concrete problems *and suggestions for solutions*.

Ok.  Some concrete stuff then.  
* Upstream should only have to create a spec file, not change stuff in
  the code, let 'require "foo"' stay 'require "foo"'. 
* Create some generizable installer, maybe assist with Package[2] or
  come up with something better, definitely useful to have a
  distutils[1]-alike system in Ruby Core IMO. 
* Create a mapping from gems to libs.  This way, we _can_ include
  RubyGems into Debian without problems.  The user can get libs that
  haven't been packaged yet or newer versions but is warned when gems
  are installed overriding a lib that already has been installed via
  dpkg and vice versa. 
  This probably sound much more easy than it is, and it's also
  more of a workaround.

Debian is about to unite much of Ruby lib/app packaging under a team. 
We are improving our system, now using setup.rb, to be able to package
and maintain a lot very efficiently.  The thing we have in common with
gems is the metadata and install part and we would very much like to
have a suiting system for that in core.  However, currently, RubyGems
increases our amount of work and we are about to go slower rather than
faster.

Paul van Tilburg
Debian Developer, Debian/Ruby (Extras) team member

1: http://www.python.org/doc/2.4.1/dist/dist.html
2: http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/144579

-- 
Student @ Eindhoven                         | email: [e-mail]
University of Technology, The Netherlands   | JID: [e-mail]
>>> Using the Power of Debian GNU/Linux <<< | GnuPG key ID: 0x50064181