[ruby-core:122232] [Ruby Feature#21311] Namespace on read (revised)

Issue #21311 has been updated by rubyFeedback (robert heiler).


First, I would like to thank mame for the explanation in regards to the namespace discussion, or rather the pre-discussion in regards to design decision(s). Probably not everyone knew about the prior discussion(s), so it seems useful to know about it.

I would also like to thank Satoshi Tagomori for the idea(s) and effort. It is actually really difficult to come up with good proposals and implementation(s). I myself wanted to have some kind of "namespace-like" feature or functionality in Ruby, but I have not been able to come up with any good proposal. It's actually not so trivial - I have a few rough ideas, but when it comes to detail, as well as implementations, there are things I was not certain, and I then gave up on it. So it is great that others manage to come up with their own ideas.

That is not to say I am all in favour of the proposal here (but, neither am I in disfavour); I actually think it is good that ideas are put forward in motion, even if these may be different ideas, or may not be "fleshed out" in detail, so to speak, in every regard. To me this is not necessarily a huge problem, because I think bugs as well as specification and documentation can be detailed at a later time too, and I think we had this with JIT/yjit/mjit/whatever-the-jit too.

Eregon is a bit skeptical and asked for some use cases; tenderlove provided one or two. I think I can also provide one or two, based on another thought process actually. I understand that these are not directly related to the namespace-suggestion here, but I do think that these may perhaps provide Eregon with another perception. Again, this is not to say that I say the proposal here is the one I would have had in mind, but I think if we look at things at a broader perspective - including refinements, and why so few people use it, despite refinements being a good idea - we may come a bit closer to useful ideas.

So perhaps, just three quick pointers, and I will leave it at that:

(1) Many years ago, on IRC, freenode, on a #mono or c-sharp channel, a C-sharp user did not like that in ruby we don't have separate "namespaces", e. g. everyone can duck-patch ("monkey-patch") things. While I think this is a useful feature (ruby being dynamic), I can somewhat understand the criticism, even if I do not fully share it. This was many many years ago, before ruby added refinements by the way. (In regards to refinements, one thing that always stopped me from using refinements was the syntax; the "use" clause in particular. But I digress.)

(2) On rubygems, we can have only one gem with a certain name, such as "configuration". I would use a module for this, e. g. "module Configuration", but of course this module would potentially clash with someone else who used the same module. And I could not host this on rubygems because a gem called configuration already exists. There are some workarounds for this, such as renaming the project, but ultimately I want to have the toplevel name "Configuration", as that makes the most sense. On github we can have multiple projects with the same name. So my thought process here was that we could perhaps add "meta-information" to classes and modules. (In my ideas, I would not have added a separate "namespace" concept, but extended classes and modules, but no matter the implementation details, I think it is more useful to think about the underlying use cases.)

For instance, ruby recently reserved "module Ruby", if I remember correctly:

https://rubygems.org/gems/ruby

Probably to be used by ruby itself. (This may have future use cases, e. g. we could
perhaps add things to it at a later time, or group certain things inside of it, but
I digress.)

If we would have meta-tag information for modules and classes, this could then allow
us to decide what to do with things we load or require. For instance, we could 
decide that "module Configuration" would be reserved rather than extended, if someone
else would use it. This would NOT be the default, but people could decide that for
their own projects - a bit like with refinements. Now, I understand that this can 
become a bit complicated (and was one reason why I did not try to propose something
like that), but if you ignore the implementation details, then I think some ruby 
developers may find it useful to be able to disallow changes made, to certain 
classes and modules, at their own discretion, for their own projects. So, not to 
handicap other users of ruby, but for their own stuff, a bit like refinements.

I can not say how useful such restrictions would be, but I think there may be a
rationale for that. I can not say how many people would actually use it (see 
refinements), but we have a somewhat similar situation with @@class_variables -
some people use it, some don't. (I don't actually; I found that it is easier to
not worry about them and just stick to @instance_variables, including on the
toplevel. Often I have just a singleton-like variable there, such as module Foo;
@use_colours = true; and go from there.)

The idea here would also be to allow rubygems to be more open about the same
name, just like github. Then we could perhaps do: "gem install fancypants --author
tenderlove" or whatever the syntax should be, if such a gem called fancypants were
to exist. (Could also be specified in a gemspec file of course.)

Of course one drawback would be that it is a bit more complicated, so one also has
to evaluate how useful this could be. But ignoring the complexity, I kind of feel
somewhat strongly about wanting to retain certain SIMPLE names, such as "module
Configuration" or "module Colours" (or module Colors; which already exists by the
way) and so forth.

(3) Last but not least, related to this partially, is the idea to add a new way
to "import" ruby code. We already have require() (including require_relative() and
load() and I think these have to remain how they are, due to backwards compatibility,
and matz also probably wants to keep this part of ruby simple. So a new way could 
perhaps be used; I am not very creative, so "import" seemed a good name, even if e. g.
python already uses it. So what would be the idea behind "import"? Well, we would keep
this extensible for the future; and it could allow for more flexibility in loading code.
For instance, "importing" into a specific namespace, and using another name, too. So if
we have:

    module Foobar

Via import-functionality, this could become "module Barfoo". Of course this is already
possible, if we assign Barfoo = Foobar; and then Foobar = nil I guess, but I found the
idea more elegant to bundle together useful things into a new import-like functionality.
And it could also offer auto-load like functionality - I don't actually mean to behave
like autoload, but to be able to "import" ruby code where it is irrelevant WHERE the
.rb file resides. Right now we often have to hardcode the path via a .rb file, and if
that is moved, downstream code can break. My idea would be to have "import" also help
us in this case, and automatically work, no matter where the .rb file is.

The idea behind import is a tiny bit related to what Satoshi Tagomori showed in regards
to:

    ns1.require('./app1') # 2048

(Though it seems to just use require, which is a bit different to my idea, but again,
that's for other discussions.)

Anyway. I don't really have an extremely strong opinion about the proposal here per se
either way; at the same time I kind of think that ruby could benefit from more thoughts in regards to larger projects, many .rb files, same-named modules and classes and so forth. This may not convince Eregon in regards to the proposal here, but I actually think that we should discuss things. Perhaps there could be a developer meeting in due time, at a later time, some months from now on, when many of the bugs were resolved, to see which ideas may work and which ones may not work. I don't know how open Satoshi Tagomori and matz are to amendments (I don't have any, mind you) but I also think it is perfectly fine to see how things go forward. IMO ruby could need some more "top-level" or "fine-tuned" control here. And making good proposals and implementing them, is a lot of work, so I think it is great that Satoshi Tagomori puts in time and effort here.

----------------------------------------
Feature #21311: Namespace on read (revised)
https://bugs.ruby-lang.org/issues/21311#change-113380

* Author: tagomoris (Satoshi Tagomori)
* Status: Assigned
* Assignee: tagomoris (Satoshi Tagomori)
* Target version: 3.5
----------------------------------------
This replaces #19744 

## Concept

This proposes a new feature to define virtual top-level namespaces in Ruby. Those namespaces can require/load libraries (either .rb or native extension) separately from other namespaces. Dependencies of required/loaded libraries are also required/loaded in the namespace.

This feature will be disabled by default at first, and will be enabled by an env variable `RUBY_NAMESPACE=1` as an experimental feature.
(It could be enabled by default in the future possibly.)

### "on read" approach

The "on write" approach here is the design to define namespaces on the loaded side. For example, Java packages are defined in the .java files and it is required to separate namespaces from each other. It can be implemented very easily, but it requires all libraries to be updated with the package declaration. (In my opinion, it's almost impossible in the Ruby ecosystem.)

The "on read" approach is to create namespaces and then require/load applications and libraries in them. Programmers can control namespace separation at the "read" time. So, we can introduce the namespace separation incrementally.

## Motivation

The "namespace on read" can solve the 2 problems below, and can make a path to solve another problem:

* Avoiding name conflicts between libraries
  * Applications can require two different libraries safely which use the same module name.
* Avoiding unexpected globally shared modules/objects
  * Applications can make an independent/unshared module instance.
* Multiple versions of gems can be required
  * Application developers will have fewer version conflicts between gem dependencies if rubygems/bundler will support the namespace on read. (Support from RubyGems/Bundler and/or other packaging systems will be needed)

For the motivation details, see [Feature #19744].

## How we can use Namespace

```ruby
# app1.rb
PORT = 2048
class App
  def self.port = ::PORT
  def val = PORT.to_s
end

p App.port # 2048

# app2.rb
class Number
  def double = self * 2
end

PORT = 2048.double
class App
  def self.port = ::PORT
  def val = PORT.double.to_s
end

p App.port # 4096

# main.rb - executed as `ruby main.rb`
ns1 = Namespace.new
ns1.require('./app1') # 2048
ns2 = Namespace.new
ns2.require('./app2') # 4096

PORT = 8080
class App
  def self.port = ::PORT
  def val = PORT.to_s
end

p App.port # 8080
p App.new.val # "8080"

p ns1::App.port # 2048
p ns1::App.new.val # "2048"

p ns2::App.port # 4096
p ns2::App.new.val # "8192"

1.double # NoMethodError
```

## Namespace specification

### Types of namespaces

There are two namespace types, "root" and "user" namespace. "Root" namespace exists solely in a Ruby process, and "user" namespaces can be created as many as Ruby programmers want.

### Root namespace

Root namespace is a unique namespace to be defined when a Ruby process starts. It only contains built-in classes/modules/constants, which are available without any `require` calls, including RubyGems itself (when `--disable-gems` is not specified).

At here, "builtin" classes/modules are classes/modules accessible when users' script evaluation starts, without any require/load calls.

### User namespace

User namespace is a namespace to run users' Ruby scripts. The "main" namespace is the namespace to run the user's `.rb` script specified by the `ruby` command-line argument. Other user namespaces ("optional" namespaces) can be created by `Namespace.new` call.

In user namespace (both main and optional namespaces), built-in class/module definitions are copied from the root namespace, and other new classes/modules are defined in the namespace, separately from other (root/user) namespaces.
The newly defined classes/modules are top-level classes/modules in the main namespace like `App`, but in optional namespaces, classes/modules are defined under the namespace (subclass of Module), like `ns::App`.

In that namespace `ns`, `ns::App` is accessible as `App` (or `::App`). There is no way to access `App` in the main namespace from the code in the different namespace `ns`.

### Constants, class variables and global variables

Constants, Class variables of built-in classes and global variables are also separated by namespace. Values set to class/global variables in a namespace are invisible in other namespaces.

### Methods and procs

Methods defined in a namespace run with the defined namespace, even when called from other namespaces.
Procs created in a namespace run with the defined namespace too.

### Dynamic link libraries

Dynamic link libraries (typically .so files) are also loaded in namespaces as well as .rb files.

### Open class (Changes on built-in classes)

In user namespaces, built-in class definitions can be modified. But those operations are processed as copy-on-write of class definition from the root namespace, and the changed definitions are visible only in the (user) namespace.

Definitions in the root namespace are not modifiable from other namespaces. Methods defined in the root namespace run only with root-namespace definitions.

## Enabling Namespace

Specify `RUBY_NAMESPACE=1` environment variable when starting Ruby processes. `1` is the only valid value here.

Namespace feature can be enabled only when Ruby processes start. Setting `RUBY_NAMESPACE=1` after starting Ruby scripts performs nothing.

## Pull-request

https://github.com/ruby/ruby/pull/13226






-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- [e-mail]
 To unsubscribe send an email to [e-mail]
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/