Re: Call for standardised package installation procedure

On Tue, Apr 08, 2003 at 10:11:28PM +0900, Gavin Sinclair wrote:
> Well, *I* prefer my OS to stick to managing hardware, processes, etc.
> That's what it's for.  SuSE, RedHat, Debian users are all using the
> same OS; they just run different applications.

I was thinking in terms of the extensive definition of OS, as not only the
kernel but also the basic tools such as package management.
Anyway, even with the definition OS == kernel, your statement doesn't
hold :-) as Debian is well on the way to running on GNU Hurd (and
later on NetBSD and FreeBSD; there were even talks about making Debian
GNU/Win32...)
 
> Some people who are unduly smug about their choice of OS (certainly
> no-one in mind from this group) would do well to remember this.
  ======

Fortunately enough as I would be the first one to blame ;-)

> Back on track, though: I see very little benefit in using generic
> lib/app management tools, as good as they are, to manage Ruby
> libraries.  Ruby libraries are very nuanced things, and benefit from
> the in-depth knowledge that Ruby has about itself and its
> configuration.  Then there's the practical problem of all the
> different installers for different distros etc.  People brush this
> aside, but it's insurmountable, IMO.

That why I think the solution is leveraging the tools available on each
platform but having Ruby drive the whole thing.

Here's one example of why I believe this to be possible. I shall
describe how I'd make it work with Debian, as it is what I know and use,
but similar things should be feasible in other systems (that's indeed
the point).

I first have to give a broad overview of how .debs are made, to later
show where we could fit Ruby. WARNING this is a bit long, skip to the
end if you don't care too much about this :)

Take for instance my .deb package of ripper-0.0.5. The tools used to
create the binary depend on having (1) some metadata and (2) a script that is
used to install everything to a temporary destination which represents
the root of the final system (a tarball is then done).

The most important part of (1) looks like this:

Source: ripper
Section: libs
Priority: optional
Maintainer: Mauricio Fern疣dez <[e-mail]>
Build-Depends: debhelper, ruby, ruby1.7, ruby-dev, ruby1.7-dev
Standards-Version: 3.5.8

Package: libripper-ruby
Architecture: any
Depends: ruby (<< 1.7), ${shlibs:Depends}
Description: Event-driven parsing of Ruby source code
 ripper is to Ruby source code what SAX is to XML.

Notice the "$(shlibs:Depends)" thing: this means that the actual
dependency will be calculated on package creation. 

As for (2), it is a Makefile with only a handful of targets (summarizing):
 * configure
 * binary
 * install
Note there's no uninstall: the OS (as in Debian, the Universal OS :-)
will take care of that.

The rules are used to install the package to a temp dir. Then the
metadata template is filled in and everything is put into an archive.
 
Now comes at last the interesting part:
take, say, the binary-indep target (binary depends on this):

binary-indep: build install
        dh_testdir -A
        dh_testroot -A
        dh_installdocs -A
        dh_installexamples -A sample/*
        dh_installchangelogs -A ChangeLog
        dh_link -A
        dh_strip -A
        dh_compress -A
        dh_fixperms -A
        dh_installdeb -A
        dh_shlibdeps -A
        dh_gencontrol -A
        dh_md5sums -A
        dh_builddeb -A

Notice all the actual operations are done through a set of tools
provided by the system.

Now imagine the following:
 * all the info is put in some standard place
 * instead of make, we use rake
 * things like dh_installexamples become cran_installexamples
 * all these "tools" are actually method calls 

Some Ruby:

sys = CRAN::current_system
task :build-indep => [:build, :install] do
  sys.installdocs
  sys.installexamples
  sys.installchangelogs
  ....
  sys.finish
end

CRAN::current_system would be an object that knows about the appropriate
procedures for the system the package is created on. But this doesn't
mean that it actually performs them, but rather that it knows what
tools are to be used. It would then, in the case of Debian, create all
the metadata and the script to make the package.

Some methods of CRAN::current_system would simply generate code for the
system's package creator; some could make more complicated things such
as finding the dependencies or correcting #!/path/to/ruby, etc...

To sum up, what I am thinking of looks like this:


(you might prefer to look at this diagram upside-down)

         Ruby Developer Side
    -------------------------------------------------------------
   |                  Ruby Meta Packaging system                 |
   |                                                             |
    -------------------------------------------------------------   <--- magic 
   |        |          |           |           |                 |     happens
   |        |          |           |           |  Ruby package   |     here
   | APT    |  urpmi   | the equiv.|  ports    |  management     |
   |        |          |  to APT   |           |  system for     |
   |        |          | on RH     |           |  platforms      |
   |        |          |  (NOT RPM)|           |  without        |
   |        |          |           |           |  high-level     |
   |        |          |           |           |   tools         |
   |        |          |           |           |                 |
   --------------------------------------------------------------
        End User side                                ^
                                                     |
                                                 means implementing
                                             APT-like functionality from
                                             scratch. rpkg would fit here 

Ruby sitting on the system-dependent toolchain, generating whatever is
needed to feed it.

Of course, all the magic is in the translator from the meta-package to
the data needed by the system tools. But it would fortunately have to be
done *only once*; e.g., no Ruby developer should have to worry about
whether the appropriate place to put documentation in is
/usr/share/doc/thepackage, /usr/doc/thepackage or
c:\ruby\docs\thepackage, he'd only write sys.installdocs *(files), and
CRAN::current_system (to continue the previous example) would know how
to either
 * do it directly
 * map it into some command for the system tools (such as "dh_installdocs -A")
   this can include more complicated things, but it always takes
   advantage of whatever the system has to offer (in win32 not that much 
   unless it's a cygwin build...)
 

> apt-get and the like should be nothing more than models for what needs
> to be achieved for Ruby library management.  I'm happy to be proven

Thet are very good models, indeed, but (IMHO always) it would be great to take
advantage of all the effort put in them. Playing nice would allow us to
 * have something working faster 
 * not have to take care of dependencies as this is done by
   APT/portage/etc
 * achieve better integration w/ the OS: nothing knows better the
   system's policies as its package management tools

> wrong, though.  Is there are precedent?  Some Perlers love the CPAN
> shell, others hate it, but does anyone use apt-get to install their
> Perl modules?

In a way yes: indirectly, when installing something (w/ APT) that required a
Perl module. I am no Perler, though :)
 
I have used APT to install Ruby modules, and sometimes go as far as
creating the .deb files only because I want deity to take care of
everything in the future. 
Example: freeride depends on ruby1.7 (imagine for some reason it won't
work with ruby 1.8), so I won't lose it by accident if I try 6 months
later to install ruby1.8 and that requires deleting first ruby1.7,
as I'll get a warning. I moreover get the "uninstall" thing for free.

Here's the list of Ruby libs I have installed using APT (the most
convenient and safest way, *always*, for me on *my system*) on this
machine (not my main one, that machine has more):

libdbm-ruby
libdbm-ruby1.7
liberuby
libfox-ruby
libfox-ruby1.7
libgd-ruby
libgtk-ruby
libmutexm-ruby
liboptparse-ruby
libpty-ruby
libpty-ruby1.7
librd-ruby
libreadline-ruby
libreadline-ruby1.7
librexml-ruby
libripper-ruby1.7
libruby
libruby1.7
libstrscan-ruby
libuconv-ruby
libxml-parser-ruby
libzlib-ruby

-- 
 _           _                             
| |__   __ _| |_ ___ _ __ ___   __ _ _ __  
| '_ \ / _` | __/ __| '_ ` _ \ / _` | '_ \ 
| |_) | (_| | |_\__ \ | | | | | (_| | | | |
|_.__/ \__,_|\__|___/_| |_| |_|\__,_|_| |_|
	Running Debian GNU/Linux Sid (unstable)
batsman dot geo at yahoo dot com

We come to bury DOS, not to praise it.
	-- Paul Vojta, [e-mail]