From: "MSP-Greg (Greg L) via ruby-core" <ruby-core@...>
Date: 2023-02-28T20:25:00+00:00
Subject: [ruby-core:112635] [Ruby master Bug#19378] Windows: Use less syscalls for faster require of big gems

Issue #19378 has been updated by MSP-Greg (Greg L).





Code using `GetFinalPathNameByHandleW` already exists in win32/win32.c, see



https://github.com/ruby/ruby/blob/c43fbe4ebd2b519601f0b90ca98fa096799d3846/win32/win32.c#L2013-L2022



For cross-reference, see also [Bug #19246 'Rebuilding the loaded feature index much slower in Ruby 3.1'](https://bugs.ruby-lang.org/issues/19246)



----------------------------------------

Bug #19378: Windows: Use less syscalls for faster require of big gems

https://bugs.ruby-lang.org/issues/19378#change-102083



* Author: aidog (Andi Idogawa)

* Status: Assigned

* Priority: Normal

* Assignee: windows

* ruby -v: 3.2.0

* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN

----------------------------------------

Hello ����



## Problem

require is slow on windows for big gems. (example: require 'gtk3'=> 3 seconds+). This is a problem for people who want to make cross platform GUI apps with ruby.



## Possible Reason

As touched on in [#15797](https://bugs.ruby-lang.org/issues/15797) it seems like require uses realpath, which is emulated on windows. It checks every parent directory. The same syscalls run many times.



## Testfile



C:\tmp\speedtest\testrequire.rb:



``` ruby

require __dir__ + "/helloworld1.rb"

require __dir__ + "/helloworld2.rb"

```



``` shell

ruby --disable-gems C:\tmp\speedtest\testrequire.rb

```



### Syscalls per File/Directory: 

1. CreateFile

2. QueryInformationVolume

3. QueryIdInformation

4. QueryAllInformationFile

5. QueryNameInformationFile

6. QueryNameInformationFile

7. QueryNormalizedNameInformationFile

8. CloseFile



### Files/Directories checked

1. C:\tmp 

2. C:\tmp\speedtest

3. C:\tmp\speedtest\helloworld1.rb

4. C:\tmp

5. C:\tmp\speedtest

6. C:\tmp\speedtest\helloworld2.rb



For two required files Ruby had to do 8*6 = **48**  syscalls.

The syscalls orginate from rb_w32_reparse_symlink_p / lstat



Rubygems live in subfolders with 9+ parts: "C:\Ruby32-x64\lib\ruby\gems\3.2.0\gems\glib2-4.0.8\lib\glib2\variant.rb"

Each file takes 8 * 9 = **72**+ calls. For variant.rb it is **80**  calls.

The result for the syscalls don't change in such a short time, so it should be possible to cache it.



With require_relative it's twice as many calls.



## Other testcases



Same result: 

``` ruby

File.realpath __dir__ + "/helloworld1.rb"

File.realpath __dir__ + "/helloworld2.rb"

```

``` ruby

File.stat __dir__ + "/helloworld1.rb"

File.stat __dir__ + "/helloworld2.rb"

```

It does not happen in $LOAD_PATH.resolve_feature_path(__dir__ + "/helloworld1.rb")







## Request 



Would it be possible to cache the stat calls when using require?

I tried to implement a cache inside the ruby source code, but failed.

If not, is there now a way to combine ruby files into one?



I previously talked about require here: [YJIT: Windows support lacking.](https://bugs.ruby-lang.org/issues/19325#note-11)



## How to reproduce 

Ruby versions: At least 3.0+, most likely older ones too.

Tested using Ruby Installer 3.1 and 3.2.

[Procmon Software by Sysinternals](https://learn.microsoft.com/en-us/sysinternals/downloads/procmon)





---Files--------------------------------

windows-no-realpath-require.patch (992 Bytes)

windows-revert-79a4484a.patch (5.42 KB)





-- 

https://bugs.ruby-lang.org/

 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/