[ruby-core:115064] [Ruby master Feature#19744] Namespace on read

Issue #19744 has been updated by deivid (David Rodr=EDguez).





>From my side, I'm happy if this feature is added because as mentioned by @h=
sbt it'd be useful for us internally.



But I'm pretty strongly against providing any fuctionality that allows reso=
lving and loading multiple versions of the same gem In Bundler/RubyGems, ev=
en if this feature could make that potentially easier. It'd still be a lot =
of work to even provide a sane specification of the feature (let alone impl=
ement it), and over the years there's been little interest in something lik=
e this. I don't believe Ruby developers miss the ability of NPM package man=
agers to do this, rather the opposite.=20



----------------------------------------

Feature #19744: Namespace on read

https://bugs.ruby-lang.org/issues/19744#change-104939



* Author: tagomoris (Satoshi Tagomori)

* Status: Open

* Priority: Normal

----------------------------------------

# What is the "Namespace on read"



This proposes a new feature to define virtual top-level namespaces in Ruby.=
 Those namespaces can require/load libraries (either .rb or native extensio=
n) separately from the global namespace. Dependencies of required/loaded li=
braries are also required/loaded in the namespace.



### Motivation



The "namespace on read" can solve the 2 problems below, and can make a path=
 to solve another problem:

The details of those motivations are described in the below section ("Motiv=
ation details").



#### Avoiding name conflicts between libraries



Applications can require two different libraries safely which use the same =
module name.



#### Avoiding unexpected globally shared modules/objects



Applications can make an independent/unshared module instance.



#### (In the future) Multiple versions of gems can be required



Application developers will have fewer version conflicts between gem depend=
encies if rubygems/bundler will support the namespace on read.



### Example code with this feature



```ruby

# your_module.rb

module YourModule

end



# my_module.rb

require 'your_module'



module MyModule

end



# example.rb

namespace1 =3D NameSpace.new

namespace1.require('my_module') #=3D> true



namespace1::MyModule #=3D> #<Module:0x00000001027ea650>::MyModule (or #<Nam=
eSpace:0x00...>::MyModule ?)

namespace1::YourModule # similar to the above



MyModule # NameError

YourModule # NameError



namespace2 =3D NameSpace.new      # Any number of namespaces can be defined

namespace2.require('my_module') # Different library "instance" from namespa=
ce1



require 'my_module' # require in the global namespace



MyModule.object_id !=3D namespace1::MyModule.object_id #=3D> true

namespace1::MyModule.object_id !=3D namespace2::MyModule.object_id

```



The required/loaded libraries will define different "instances" of modules/=
classes in those namespaces (just like the "wrapper" 2nd argument of `Kerne=
l.load`). This doesn't introduce compatibility problems if all libraries us=
e relative name resolution (without forced top-level reference like `::Name=
`).



# "On read": optional, user-driven feature



"On read" is a key thing of this feature. That means:



* No changes are required in existing/new libraries (except for limited cas=
es, described below)

* No changes are required in applications if it doesn't need namespaces

* Users can enable/use namespaces just for limited code in the whole librar=
y/application



Users can start using this feature step by step (if they want it) without a=
ny big jumps.



## Motivation details



This feature can solve multiple problems I have in writing/executing Ruby c=
ode. Those are from the 3 problems I mentioned above: name conflicts, globa=
lly shared modules, and library version conflicts between dependencies. I'l=
l describe 4 scenarios about those problems.



### Running multiple applications on a Ruby process



Modern computers have many CPU cores and large memory spaces. We sometimes =
want to have many separate applications (either micro-service architecture =
or modular monolith). Currently, running those applications require differe=
nt processes. It requires additional computation costs (especially in devel=
oping those applications).



If we have isolated namespaces and can load applications in those namespace=
s, we'll be able to run apps on a process, with less overhead.



(I want to run many AWS Lambda applications on a process in isolated namesp=
aces.)



### Running tests in isolated namespaces



Tests that require external libraries need many hacks to:



* require a library multiple times

* require many different 3rd party libraries into isolated spaces (those ma=
y conflict with each other)



Software with plugin systems (for example, Fluentd) will get benefit from n=
amespaces.



In addition to it, application tests can avoid unexpected side effects if t=
ests are executed in isolated namespaces.



### Safely isolated library instances



Libraries may have globally shared states. For example, [Oj](https://github=
.com/ohler55/oj) has a global `Obj.default_options` object to change the li=
brary behavior. Those options may be changed by any dependency libraries or=
 applications, and it changes the behavior of `Oj` globally, unexpectedly.



For such libraries, we'll be able to instantiate a safe library instance in=
 an isolated namespace.



### Avoiding dependency hells



Modern applications use many libraries, and those libraries require much mo=
re dependencies. Those dependencies will cause version conflicts very often=
. In such cases, application developers should resolve those by updating ea=
ch libraries, or should just wait for the new release of libraries to confl=
ict those libraries. Sometimes, library maintainers don't release updated v=
ersions, and application developers can do nothing.



If namespaces can require/load a library multiple times, it also enables to=
 require/load different versions of a library in a process. It requires the=
 support of rubygems, but namespaces should be a good fundamental of it.



## Expected problems



### Use of top-level references



In my expectation, `::Name` should refer the top-level `Name` in the global=
 namespace. I expect that `::ENV` should contain the environment variables.=
 But it may cause compatibility problems if library code uses `::MyLibrary`=
 to refer themselves in their deeply nested library code.



### Additional memory consumption



An extension library (dynamically linked library) may be loaded multiple ti=
mes (by `dlopen` for temporarily copied dll files) to load isolated library=
 "instances" if different namespaces require the same extension library. Th=
at consumes additional memory.



In my opinion, additional memory consumption is a minimum cost to realize l=
oading extension libraries multiple times without compatibility issues.



This occurs only when programmers use namespaces. And it's only about libra=
ries that are used in 2 or more namespaces.



### The change of `dlopen` flag about extension libraries



To load an extension library multiple times without conflicting symbols, al=
l extensions should stop sharing symbols globally. Libraries referring symb=
ols from other extension libraries will have to change code & dependencies.



(About the things about extension libraries, [Naruse also wrote an entry](h=
ttps://naruse.hateblo.jp/entry/2023/05/22/193411).)



# Misc



The proof-of-concept branch is here: https://github.com/tagomoris/ruby/pull=
/1

It's still work-in-progress branch, especially for extension libraries.









--=20

https://bugs.ruby-lang.org/

 ______________________________________________
 ruby-core mailing list -- [e-mail]
 To unsubscribe send an email to [e-mail]
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-c=
ore.ml.ruby-lang.org/