Re: Core support for Gems, and namespace

On Wed, 28 Jul 2004 00:14:29 +0900, Luke A. Kanies <[e-mail]> wrote:
> On Tue, 27 Jul 2004, Austin Ziegler wrote:
>> Here's what I do: I package both as a gem and as a .tar.gz file.
>> I have taken the installation script written by Dave Thomas for
>> an old version of RDoc and made significant changes. I have a
>> standard library format:
> [SNIP]
> Yeah, see, I exactly don't want to duplicate my work like that.
> It's not that the process is that onerous, but I'd like there to
> be a single good solution.

There isn't one -- yet. An alternative solution is also being worked
on. It's called "RPA", and is intended to be the Debian or ports of
Ruby. I'm not sold on that one, either, because it's not a single
installation mechanism that developers can use, but more of a
process that also has an installation mechanism. The packagers
aren't necessarily the developers.

I provide both because I think that both are viable and will be for
some time to come. As I said, it's pretty much automatable (if you
look at my install.rb, there's basically nothing specific to a given
installation; that's handled entirely by my library layout). I need
to do a few things to it (e.g., look for collision detection and
make sure that we're only installing on our ancestral versions if at
all possible, as well as make an uninstall file).

>> That said, I'm not sold on gems. [...] There are some good things
>> about RubyGEMs, but I think that there are some real questions
>> about overengineering here.
> I don't have a problem adding features that will make sense when
> the ruby community is more mature; my biggest problem was the idea
> of being forced to specify whether I wanted a gem or not, and the
> documentation on the rubygems page still claims (in all places I
> could find) that that is required. The docs obviously need to be
> updated.

Much of this is discussed far more often and in far greater depth on
ruby-talk and the RubyGems lists (and I'm not on the latter).

>> I don't have anything really specific, but there is a part of me
>> that finds certain parts of RubyGems uncomfortable, and I'm not
>> really sure what it is. The metadata is nice, and that it's a
>> packaged format is also nice.
> I guess I don't really understand why it doesn't just add the gems
> directory to the ruby search path and be done with it. That seems
> very easy:  It gets total control over a directory, and it just
> needs to make sure its directory is added to ruby's global search
> path, although I don't actually know if (like perl) ruby loads a
> config.rb file to get its search path.

[...]

> This is the part that concerned me; I can't find any mention of
> this feature on the gems website. And what's up with the
> difference in name between the gem and the library? Is that going
> to be a common problem? Very strange.

That's been answered a couple of times, and it's versioning. Install
a couple of gems on your system and you'll see what I mean. The
directory path looks like:

  lib/ruby/gems/VERSION/gems/diff-lcs-1.0.1/...
  lib/ruby/gems/VERSION/gems/diff-lcs-1.0.2/...

Which version should actually be installed in the default Ruby path?
This is what require_gem does, in part, and the installed stub loads
the version that is greatest in version number. (I just tried this
by installing 1.0.2 before 1.0.1.) You must specify the *gem* name
rather than the default load library (e.g., diff-lcs rather than
diff/lcs) because the gem name is all that RubyGems knows a library
by. This is one of the things that I don't like about RubyGems and
think that RPA will possibly handle better -- but I'm not sure.

See, the ultimate problem I have with the versioning scheme is that
NO ONE should EVER be using diff-lcs-1.0.1 anymore because there's
some ugly bugs in it. When I release diff-lcs-1.1.0, people could
theoretically use 1.0.2, but should use 1.1.0. The only time that I
think that having multiple versions of a library installed on your
system makes sense is when the API has changed (e.g., diff-lcs-1.x.y
vs. diff-lcs-2.x.y).

I want an installation system that gives me some of the nice
features that RubyGems has (remote installation, etc.) but doesn't
actually do some of the versioning stuff that RubyGems does, because
I really don't believe that it will be an issue for some time to
come with Ruby -- Ruby the language is changing rapidly right now,
too.

>> I think that the namespace issue is something best left for a
>> later day, to be quite honest. In the two years that I've been
>> using Ruby, there have been two namespace collisions that I've
>> seen.
> The problem with leaving it until later is that it will take much
> more effort [than] to resolve now. If we have namespace standards
> now, then we will already have a mechanism for resolving
> collisions when they do crop up. And my concern wasn't just for
> collisions: I also don't want to be in a situation where 95% of my
> libraries are installed directly at the top-level of the library
> path.

Okay, who is going to impose those standards? Who is going to come
up with them in the first place? Where does something like
progressbar.rb fit (it's a textual progressbar)? I don't want to go
the direction of Java domain-namespaces (e.g.,
celsoft.com/interface), and I don't want them unnecessarily deep.
(ui/text/progressbar => UI::Text::Progressbar). Perl's require
equivalent doesn't use filenames, but namespaces. Thus, you can only
ever HAVE one item that provides Text::Format. Ruby, however, could
have text/format/austin.rb and text/format/joe.rb which both provide
competing implementations of Text::Format. Not nice, for certain,
but certainly feasible.

It's been a while since I've done any serious Perl programming, as
well, but I don't think that modules/packages are open in Perl the
way that they are in Ruby. Thus, we'll have ext/numeric/format.rb
(from the extensions project) which actually modifies the Numeric
class. The namespace problem in Ruby is completely different than
the namespace problem in any other language that I know, and is
usually focussed on imported method names more than library
collisions.

[...]

> If people try to install ruby scripts but they have to spend two
> hours hunting down libraries to make them work, then they'll just
> sit on their hands until a perl solution shows up (which isn't
> likely to happen with the stuff I'm working on). Net::LDAP in perl
> is a good example -- I use that library in almost all of my perl
> coding, and it requires tons of other libraries, but I know that
> people can just use CPAN to install it and all of its prereqs and
> it'll just work.

Unless, of course, you get caught in CPAN upgrade hell. I have had
this happen to me. RubyGems is trying to avoid this. It's the one
place that I can see the utility of the versioning scheme (it allows
a library that requires another library to depend on a specific
version and with the right gemspec, that version will be installed
without affecting the general case). RPA will be doing the same
thing in concept.

> As to perldoc sucking, I'm not convinced. Like I said, every perl
> script I've written for the last 3 years or so has embedded
> documentation, and it's set up to both provide a usage statement
> and a full man page, including one that can be converted to a
> system man page. It's a standard, and everything -- including CPAN
> -- supports it.

*shrug* Perldoc sucks as a format. The tools around perldoc are
pretty cool.

> As to Ruby's "solution", I think you mean "solutions". I forgot
> about 'ri', but that gets added to 'rd' and 'RDoc', and all of
> them seem to be decent at documenting programming interfaces
> (although they also seem to all prefer html) and not good at
> documenting scripts themselves. Apparently 'man' pages are not
> popular anymore, but I love the fact that all of my perl libraries
> get their docs installed as man pages, and I can just say 'perldoc
> <lib> ' and get the equivalent of a man page.

No, I mean solution. rd is effectively gone. ri is generated from
rdoc. If someone wanted to, they could easily write an rdoc
formatter to create groff manpages and the installers could happily
use that, too. I think that RubyGems has to do some work with
respect to ri integration, but ri is the preferred way for handling
this stuff at least right now. As far as manpage/help stuff is
concerned, there are two ideal choices for option handling and
parsing, both of which will give you awesome help output. The first
is optparse and the other is a port of a Perl library (can't
remember the name, offhand) by GGaramuno.

> Is one of these three solutions now accepted as the 'standard'?
> How do they relate? I now see that RDoc and ri are both part of
> the same sourceforge project, which in some ways just confuses me
> more... The project page doesn't have any documentation, so that
> doesn't really help.

rdoc and ri are now both part of the Ruby sources. Most of this
information has been communicated both on ruby-core and on ruby-talk
in the last six months.

>> If you want a copy of my install.rb, you can grab it from the
>> Diff::LCS package or CVS.
> See, that's the thing: I don't know what I should do to package my
> libraries in order to make them simpler for other people to use
> them and keep them updated. Yes, I can provide an install.rb, but
> that's not much better than the configure/make/make install cycle.

The thing is, the configure/make/make install cycle persists because
it works.

-austin
-- 
Austin Ziegler * [e-mail]
               * Alternate: [e-mail]