Re: Core support for Gems, and namespace

On Thursday, July 29, 2004, 1:44:03 AM, Austin wrote:

> On Wed, 28 Jul 2004 23:00:34 +0900, Gavin Sinclair
> <[e-mail]> wrote:
>> On Wednesday, July 28, 2004, 9:51:42 PM, Austin wrote:
>>> 1. The binary and site_lib stubs breaks the versioning concept,
>>> and
>> That's mostly false. The binary stub loads the most recently
>> installed version of the app. This is not an ideal situation,
>> perhaps, but it's not terrible either. (Chad has thought about
>> adding to the stub so you can do, for instance, "rake --version
>> 0.3.2 ...".)
>> 
>> The site_ruby stubs, however, do not break anything. They provide
>> a mechanism by which you can access a gem library through the
>> normal 'require' mechanism. 'require' knows nothing about
>> versions, so nothing is broken. This is meant to be a convenience.
>> If you choose to use 'require', you're getting exactly what you
>> want.

> Um, you just made my point for me on both counts. The convenience
> stubs in site_ruby *do* break the versioning features of RubyGems.
> Don't get me wrong, they will help increase the usability of
> RubyGems, but they also remove the version control capabilities
> through their use.

OK, that's a fair way of looking at it.  I just figured that it's a
developer's choice to use 'require', which means they have no hope of
any versioning.  That's not breaking anything; it's just a feature
that's not there.

> The binary stub does load the most recently installed version. It
> loads the most current version, but that's wrong, as the ldiff that
> gets installed with diff-lcs-1.0.1 is broken. Yes, I did a test as a
> deliberate borking of things, but if one installs 1.0.2 and then
> installs 1.0.1, require_gem by itself will load the 1.0.2 version,
> but the ldiff installed will use the most recently installed
> version.

I think I'd prefer the app stub to load the most current version by
default, as would you, it seems.  Running an old application doesn't
seem all that gratifying.

>>> there are other problems with the use of them from the developer/
>>> deployment perspective. (I had to change ldiff in diff-lcs to not
>>> use the "if __FILE__ == $0" common feature. It is expected that
>>> the loaded file will return an error code -- and not all "bin"
>>> files are written that way.)
>> I doubt there's a good reason to use "if __FILE__ == $0" in an
>> application. An app is, after all, explicitly designed to be run.
>> That idiom makes sense for libraries, not (AFAICS) apps.

> *shrug* I try to write my apps in such a way that they could be
> wrapped by another app. Thus, I had used the execution block as:

>   if __FILE__ == $0
>     exit Diff::diffprog(ARGV, $stdout, $stderr)
>   end

Yeah, not a bad design, I guess.

>>> 2. The disconnect between "require_gem 'diff-lcs'" and the nominal
>>> "require 'diff/lcs'" is more than annoying, it's problematic.
>> It might be a good idea to name your package 'diff-lcs', then. I know
>> I'm using a stupid argument here, as RubyGems has certainly robbed you
>> of some flexibility in this respect. If it were my library, I'd
>> probably choose that name anyway. (Hey, this thread is partially about
>> namespaces, right?)

> Okay; I think you're not understanding what I'm saying here. To
> clarify: the package is diff-lcs. The gem names are diff-lcs-1.0.1
> and diff-lcs-1.0.2 and will be diff-lcs-1.1.0. To use a specific
> version, rather than doing

>   require "diff/lcs"

> which is the preferred mechanism, users must remember the package
> name:

>   require_gem "diff-lcs"

What I'm saying is that you could arrange it so that users do this:

  require 'diff-lcs'

and

  require 'diff-lcs/array'

I know that's not your preference, but it would probably be mine if I
were writing the library, whether RubyGems existed or not.

> Ideally, there should be a way to map the require specification to
> the package name, and it's the gem's responsibility to do this, IMO.
> What I *want* is:

>   require_gem "diff/lcs", "= 1.0.2"

> to be interpreted as if it were:

>   require_gem "diff-lcs", "= 1.0.2"

That sounds like a bad idea to me.  Your package does demonstrate an
area where RubyGems could be improved, I think, but having yet another
level of indirection seems flawed.  When you do 'require_gem X', that
X should be the same name as the gem - it's simply what the command is
saying.

> If that were to happen, then there'd be absolutely no reason for me
> to not use require_gem in my top-level requires.

That's surprising.  require_gem won't work in non-RubyGems-enabled
platforms, so there's a reason for you.

>>> It means that I can really only provide one loading mechanism for a
>>> given library, whereas I might theoretically have more than one.
>>> [...]
>> Well, what you have to do is this:
>> 
>> require_gem 'diff-lcs'
>> require 'diff/lcs/array'
>> 
>> You are obviously relying on the site_ruby stub, [...]

> Not quite. I have designed my library. I should not have to redesign
> it in any way simply because of the packaging mechanism.

RubyGems is not just a packaging mechanism, remember.  RubyGems is
justified in demanding some cooperation from libraries that use it.
I don't think it is onerous, but again, your case does show some ways
it could be better.

> As RubyGems stands now, I have to do a bit *too much* work. (In
> fact, diff-lcs-1.0.1 existed entirely because of problems with
> RubyGems.) The ability to have multiple stubs will help
> significantly.

Yeah, I see that as an inevitable feature.  I'm just more focused on a
couple of other aspects of the gemspec format at the moment.

> Many of the issues I have will be eliminated if RubyGems is
> integrated into Ruby directly, as I won't have to use a separate
> command to require a gem.

Quite right.  Hopefully that will one day be the case.

>> So now (with RubyGems CVS) you can do this:
>> 
>> require_gem 'extensions/string'
>> 
>> which is exactly shorthand for the above two lines. However, this
>> only "works" if your gem is named the same as your project directory.

>> What remains "unsolved" here is your desire to name a gem with a slash
>> in it. That's not allowed. Perhaps it could be. Hopefully Chad and
>> Rich will think about it.

> No, you're not understanding. I don't want a gem with a slash. My
> gem is diff-lcs. My *package* is diff-lcs. But the *namespace* is
> Diff::LCS, therefore the require is of diff/lcs.

That "therefore" is not necessarily justified.  The require could be
'diff-lcs' instead, and things would work more nicely (as in,
require_gem 'diff-lcs/array').  I can think of objective arguments
both ways.

When each package has its own filesystem space, as RubyGems allows,
then the directory format doesn't have all the same connotations as it
does in a shared filesystem space (site_ruby/require).

>>> 3. Certain types of changes should not be permitted to exist
>>> simultaneously (e.g., diff-lcs-1.0.1 and diff-lcs-1.0.2), whereas
>>> others should be acceptable (e.g., the upcoming diff-lcs-1.1.0 vs
>>> diff-lcs-1.0.2). This can't be solved by RubyGems alone, but
>>> there is no way for me, as a developer, to mark older gems as
>>> obsolete and incompatible with earlier versions.
>> Why shouldn't they be allowed to co-exist? If someone writes a
>> really cool app that depends on diff-lcs-1.0.1, and they've thoroughly
>> tested it, then what's wrong with that? It may not be the way I would
>> package it, but if they're happy with 1.0.1, why should they be forced
>> into an upgrade cycle controlled by you?

> For the same reason that security fixes should obsolete broken
> libraries. You don't want broken libraries in use by anyone.

No you don't.  But you can't do anything about it.  It's certainly not
RubyGems's job to do so.  Once people have a library installed, it's
there until they choose to upgrade.

> (And the bug in diff-lcs-1.0.1 essentially makes it unusable; there
> is no way to actually build an app that depends on certain features
> because the breaks are internal and prohibit the emission of the
> information required.)

If it's that bad, then no-one would release a library or app that is
based on it, so there's no problem.

> Remember, I don't have a problem with 1.1.0 and 1.0.2 coexisting;
> however, 1.0.1 is something that should (1) be removed from the gem
> repository and (2) be removed when upgrading to 1.0.2.

Too draconian (as an automatic measure).  The points that you make
about specifying "= 1.0" and resolving to "= 1.0.*" (snipped) are
good, but versioning policies are something for humans to argue about,
not for RubyGems to enforce.  If I write an app that I've positively
tested with all diff-lcs versions from 1.0.2 to 1.4.0, who are the
versioning policy police to argue?

>> It sounds like a gem-only version of Ruwiki would run like a
>> dream. Not that I'm suggesting that :)

> Mmmm. I'm not convinced. One concern is that ruwiki_servlet
> currently gets its data path as "./data/". [...]

Ah yes, that sort of issue is a difficult one.

Anyway, I'm sure you'll be happy when the multiple stubs feature is
done.

Cheers,
Gavin