[ruby-core:97835] [Ruby master Misc#16778] Should we stop vendoring default gems code?

Issue #16778 has been updated by deivid (David Rodr=EDguez).


Yeah! You're right about those other reasons. Specially for security issues=
 in standard libraries, gemification is great. I believe those benefits rem=
ain with my proposal though.

I didn't mention any drawbacks because I couldn't think of any. Some stuff =
came to my mind, but in the end, I'm not sure they are really drawbacks. Fo=
r example:

* ruby-core developers will no longer be able to add changes directly to de=
fault libraries in the ruby-core repo. That's true, but this seems like an =
improvement to me. In the previous situation, they would also need to go to=
 the upstream repo and propose those same changes. And if they forget to do=
 that, merging the upstream repo back into ruby-core would revert their cha=
nges and make the original issue reappear. This has happened in the recent =
past, and normally due to issues related to the different structure of the =
repositories. All these issues would be gone with my proposal.

* There will be some extra download of code from the network when some make=
 targets are first run after cloning the repo. Well, this is true, but ther=
e will be less code downloaded when ruby is cloned because all the default =
libraries will no longer be vendored. So I don't really think this is a dra=
wback either.

Hope this made my proposal a bit more clear.

So, to answer the question of "what would actually be changed as a conseque=
nce of this"?

* User facing changes: none. I would expect the result of `./configure && m=
ake && make install` to be exactly the same. This is a proposal to change o=
nly how default gems are managed _internally_.

* Developer facing changes: minimal, and should be aimed to make the ruby-c=
ore developer experience better. The idea is that:
  * There will be no more need of default gem syncronization bringing code =
from upstream into ruby-core's source control. So, less work.
  * There will be no more need to test default gems inside ruby-core repo. =
So lighter local tests, lighter CI.
  * No more code duplication between ruby-core and upstrem repos. So, less =
code duplication, less confusion for contributors.
  * Any other ruby-core developer local workflows other than these things s=
hould remain unchanged.

----------------------------------------
Misc #16778: Should we stop vendoring default gems code?
https://bugs.ruby-lang.org/issues/16778#change-85057

* Author: deivid (David Rodr=EDguez)
* Status: Open
* Priority: Normal
----------------------------------------
Currently ruby-core vendors all the code in default gems, and runs the test=
s for each of them.

Also, ruby-core continuously updates the vendored code of default gems to s=
ync with the upstream repos. That's overhead work, not only from syncronizi=
ng the code itself, but it also requires perfect syncronization of releases=
 to avoid including versions of default gems that are different from releas=
ed versions.

Also, this causes confusion for contributors because the code lives "duplic=
ated" in two different places. Some times contributors will open a PR in th=
e ruby-core repo, only to find out that they need to go to the upstream rep=
o and contribute it in there. And this rule is not even always followed and=
 sometimes ruby-core contributors apply patches to the vendored code direct=
ly (many times to fix test-only issues inherent to the different structure =
of the core repository). These patches then need to be contributed back to =
the upstream repo.

I believe that all of that kind of defeats the point of "gemification" of t=
he standard library.

Once some ruby code its gemified, it should be the new upstream's responsab=
ility to make sure the code works and it's properly tested, and ruby-core s=
hould be free'd from that responsability.

Maybe ruby-core could do something along the following lines:

* Remove all the vendored code from default gems.
* When this code is needed for internal tests, manage it as a development d=
ependency, clone it as necessary on non source controlled locations, and us=
e it from there.
* Maybe a file similar to `gems/bundled_gems` can be added for default gems=
 indicating their versions and upstream repos, to ease things.
* Upon `make install`, clone the proper version of each default library and=
 get it installed in the default $LOAD_PATH.
* Maybe add some bare high level CI checks to ensure that all default libra=
ries can be properly required after `make install`, and that their executab=
les (if they include any) can also be run.

This should bring several benefits to the development process:

* No more duplicated code.
* No more syncronization from upstream to ruby-core.
* No more syncronization from ruby-core to upstream.
* No more confusion around the canonical place to contribute.
* No more complexities derived from the different organization of the code =
depending on whether it lives in ruby-core or outside.  =


I believe jruby already does something like this so it'd be interesting to =
get some input from them.

If this is a direction the ruby-core team would like to take, I'm happy to =
help @hsbt with small steps towards slowly approaching to this high level g=
oal.



-- =

https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=3Dunsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>