From: SASADA Koichi <ko1@...>
Date: 2012-07-31T21:29:46+09:00
Subject: [ruby-core:46896] (Half-baked DRAFT) new `require' framework

Hi,

This is `pre'-feature request.  I can't make up complete feature
request.  I post it to summaries current consideration.

----------------------

= Abstract

Now, `require' only loads `.rb' script or `.so' files from `file system'.

(Proposal-1)
This proposal extends the current limitation of require:

  (1) load script which is represented by other than `.rb' and `.so'.
    (example: pre-compiled file, encrypted file)

  (2) load script from location from other than `file system'.
    (example: require from zip archive, require over network,
     require files provided by gem efficiently using file location db)

(Proposal-2)
However, Proposal-1 is too big and we have not-enough time to discuss
about it.  I propose alternative feature: `require' contractible
primitives.  Break down `require' process and add primitives which can
implement `customized require' in Ruby (or C level).
Using (Proposal-2), we can make gem which provides (Proposal-1).  We can
try and discuss about good design of (Proposal-1).


= Background

Now, `require' only loads `.rb' script or `.so' files from `file system'.

(Problem example 1)
We can't specify .rb file in a zip archive, a tar archive, and so on.

(Problem example 2)
If `n' gems are installed, `n' paths are added to $LOAD_PATH by default.
 If `n' is large, then performance of `require' slows down heavily.

(Problem example 3)
Pre-compile (AOT: Ahead-of-Compile) is known technique to reduce loading
time.  However, we don't have any way to load data other than `.so' and
`.rb'.
AOT compiler `CastOff' <https://github.com/soba1104/CastOff> compile .rb
file into .so file.  It is a few way to make an AOT compiler.  However,
making .rb files into .so files increase binaries total size (about 10
times larger).


== Related work

=== JRuby

JRuby can require file from `.jar' (and .war file? sorry i'm not sure).
 JRuby also extends file path like VFS (virtual file system).  JRuby can
open a file in a .jar file like file system.
(JRuby GURU: Please complement this section!)

=== Rubinius

Rubinius supports AOT compile (a ruby script to a compiled file).
(Rubinius GURU: Please complement it!)

=== Python: PEP-273, PEP-302

Python can import file from Java file (described in PEP-273).
Python extends PEP-273 to import any format file by PEP-302.
This proposal (Proposal1) is based on PEP-302.

However, PEP-302 can only support to import `.py' and `.pyc' files, not
other file format.

=== Java Jigsaw project <http://openjdk.java.net/projects/jigsaw/>

(Java GURU: Help me to complement it!)

=== Other languages

any other similar interfaces?

= Proposal (1)

Make an `extensible require' framework for require().
It's my draft (not implemented yet):
  <http://www.atdot.net/sp/view/tcy08m/readonly>

== Interface description

(snip... not described yet)

== Usecase

(snip... not described yet)

= Proposal (2)

(snip... not described yet, sorry ;-)

= Discussion

== (Proposal-1) (1) is needed?
  (Python's PEP-302 doesn't support it.  It seems overkill)

== Need VFS?

We can't introduce VFS in 2.0 (because there are no enough time to
introduce it.

== Other considerations?


----

As you can see, it is only draft.
I send it because today is last day of July :)
(Aug is dead-line of big-feature for Ruby 2.0)

Thanks,
Koichi

-- 
// SASADA Koichi at atdot dot net