[ruby-core:121876] [Ruby Feature#21311] Namespace on read (revised)

Issue #21311 has been updated by tagomoris (Satoshi Tagomori).


I'm sorry that I skipped to make effort to describe the motivation and copied the motivation section from the past ticket (and it was not well described enough).
Let me explain the motivation in detail. I had some mixed points below:

### Being able to namespace existing code and libraries

By using namespace, we can mount two, or more, dozens of different Rack applications on a Rack app server.
The use cases of this feature are:

* Mounting server-less applications (run on AWS Lambda, separated on processes in production) on a single app server for development
  * Apps may have conflicting classes (like `User`)
  * Apps may have conflicting dependencies
  * Apps need to load different set of environment variables
  * (This is my original motivation to development namespace)
* Mounting two different revisions of an application for useful deployments (older and newer commits of an app)
  * App server can implement in-process blue-green deployment
  * App server can check diffs of responses of two revisions from a (duplicated) request
  * App server can compare response time of two revisions from a (duplicated) request
* Mounting two different set of dependency versions of an application
  * This is a kind of blue-green deployments, but for dependencies without any app code diffs

The first one is to update my app server LFA (https://github.com/tagomoris/LFA). And I believe I would be very positive to try implementation of the second/third one if I were maintaining web apps written in Ruby.

### Avoiding unexpected globally shared modules/objects

My case of this is Fluentd and Oj. Oj has a global set of configuration `Oj.default_options`. It can be set from anywhere, and it can break the behavior globally.
Many of Fluentd plugins use Oj, and the behavior of Oj may be changed by options set by different plugins.
It seems a kind of bug and should not happen, but Oj still doesn't have per-instance configuration APIs.

### Multiple versions of gems can be required

Fluentd too. Plugins are different and independent software developed by many different maintainers, having different sets of dependencies, but those plugins can be loaded on a Fluentd process.
In my opinion, any other software having plugin systems should have same problem potentially.

### Namespace monkey patches

It may not happen frequently now, but potentially happens, and (in my opinion) it prevents us to implement Ruby core feature in Ruby.
I'm unsure how it actually prevents Ruby committers, but if built-in Ruby code are protected from monkey patches by namespace, we can implement many things in Ruby.

For example, require/load mechanism is implemented in C, but most of those code are just doing path construction (string operations) and file system accesses. It's really obvious that Ruby is much better language than C in such purpose. And now, it can be JIT-ed!

I'm really unsure how monkey patch risk prevents committers to do those things, so this section can be ignorable :P

----------------------------------------
Feature #21311: Namespace on read (revised)
https://bugs.ruby-lang.org/issues/21311#change-112945

* Author: tagomoris (Satoshi Tagomori)
* Status: Open
----------------------------------------
This replaces #19744 

## Concept

This proposes a new feature to define virtual top-level namespaces in Ruby. Those namespaces can require/load libraries (either .rb or native extension) separately from other namespaces. Dependencies of required/loaded libraries are also required/loaded in the namespace.

This feature will be disabled by default at first, and will be enabled by an env variable `RUBY_NAMESPACE=1` as an experimental feature.
(It could be enabled by default in the future possibly.)

### "on read" approach

The "on write" approach here is the design to define namespaces on the loaded side. For example, Java packages are defined in the .java files and it is required to separate namespaces from each other. It can be implemented very easily, but it requires all libraries to be updated with the package declaration. (In my opinion, it's almost impossible in the Ruby ecosystem.)

The "on read" approach is to create namespaces and then require/load applications and libraries in them. Programmers can control namespace separation at the "read" time. So, we can introduce the namespace separation incrementally.

## Motivation

The "namespace on read" can solve the 2 problems below, and can make a path to solve another problem:

* Avoiding name conflicts between libraries
  * Applications can require two different libraries safely which use the same module name.
* Avoiding unexpected globally shared modules/objects
  * Applications can make an independent/unshared module instance.
* Multiple versions of gems can be required
  * Application developers will have fewer version conflicts between gem dependencies if rubygems/bundler will support the namespace on read. (Support from RubyGems/Bundler and/or other packaging systems will be needed)

For the motivation details, see [Feature #19744].

## How we can use Namespace

```ruby
# app1.rb
PORT = 2048
class App
  def self.port = ::PORT
  def val = PORT.to_s
end

p App.port # 2048

# app2.rb
class Number
  def double = self * 2
end

PORT = 2048.double
class App
  def self.port = ::PORT
  def val = PORT.double.to_s
end

p App.port # 4096

# main.rb - executed as `ruby main.rb`
ns1 = Namespace.new
ns1.require('./app1') # 2048
ns2 = Namespace.new
ns2.require('./app2') # 4096

PORT = 8080
class App
  def self.port = ::PORT
  def val = PORT.to_s
end

p App.port # 8080
p App.new.val # "8080"

p ns1::App.port # 2048
p ns1::App.new.val # "2048"

p ns2::App.port # 4096
p ns2::App.new.val # "8192"

1.double # NoMethodError
```

## Namespace specification

### Types of namespaces

There are two namespace types, "root" and "user" namespace. "Root" namespace exists solely in a Ruby process, and "user" namespaces can be created as many as Ruby programmers want.

### Root namespace

Root namespace is a unique namespace to be defined when a Ruby process starts. It only contains built-in classes/modules/constants, which are available without any `require` calls, including RubyGems itself (when `--disable-gems` is not specified).

At here, "builtin" classes/modules are classes/modules accessible when users' script evaluation starts, without any require/load calls.

### User namespace

User namespace is a namespace to run users' Ruby scripts. The "main" namespace is the namespace to run the user's `.rb` script specified by the `ruby` command-line argument. Other user namespaces ("optional" namespaces) can be created by `Namespace.new` call.

In user namespace (both main and optional namespaces), built-in class/module definitions are copied from the root namespace, and other new classes/modules are defined in the namespace, separately from other (root/user) namespaces.
The newly defined classes/modules are top-level classes/modules in the main namespace like `App`, but in optional namespaces, classes/modules are defined under the namespace (subclass of Module), like `ns::App`.

In that namespace `ns`, `ns::App` is accessible as `App` (or `::App`). There is no way to access `App` in the main namespace from the code in the different namespace `ns`.

### Constants, class variables and global variables

Constants, Class variables of built-in classes and global variables are also separated by namespace. Values set to class/global variables in a namespace are invisible in other namespaces.

### Methods and procs

Methods defined in a namespace run with the defined namespace, even when called from other namespaces.
Procs created in a namespace run with the defined namespace too.

### Dynamic link libraries

Dynamic link libraries (typically .so files) are also loaded in namespaces as well as .rb files.

### Open class (Changes on built-in classes)

In user namespaces, built-in class definitions can be modified. But those operations are processed as copy-on-write of class definition from the root namespace, and the changed definitions are visible only in the (user) namespace.

Definitions in the root namespace are not modifiable from other namespaces. Methods defined in the root namespace run only with root-namespace definitions.

## Enabling Namespace

Specify `RUBY_NAMESPACE=1` environment variable when starting Ruby processes. `1` is the only valid value here.

Namespace feature can be enabled only when Ruby processes start. Setting `RUBY_NAMESPACE=1` after starting Ruby scripts performs nothing.

## Pull-request

https://github.com/ruby/ruby/pull/13226






-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- [e-mail]
 To unsubscribe send an email to [e-mail]
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/