[ruby-core:40917] [ruby-trunk - Feature #5562] Improvement of Windows IO performance

Issue #5562 has been updated by Luis Lavena.

Category set to core
Assignee set to Usaku NAKAMURA

Hiroshi Shirosaki wrote:
> I'm not sure how to do with test-all. mingw ruby seems not to pass all tests.

That is correct, if you add skip to the following two:

TestRequire#test_require_invalid_shared_object
TestRubyOptions#test_segv_test

make test-all will complete

> Anyway I tried to make test-all.
> If ruby-core committers like this result, I hope this patch merged to trunk.
> 
> I have modified a patch to trunk. Previous patch conflicts with trunk. I have resolved and added minor fixes.
> 
> make test-all result become better.
> 

The following is the list of tests that fails (using RubyInstaller's base) trunk versus your patch:

trunk:
TestArgf#test_textmode
TestDir_M17N#test_filename_bytes_euc_jp
TestDir_M17N#test_filename_euc_jp
TestDir_M17N#test_filename_ext_euc_jp_and_int_utf_8
TestDir_M17N#test_filename_extutf8
TestDir_M17N#test_filename_extutf8_inteucjp_representable
TestDir_M17N#test_filename_extutf8_inteucjp_unrepresentable
TestDir_M17N#test_filename_utf8_raw_name
TestGDBM#test_reorganize
TestGDBM#test_s_open_create_new
TestGDBM#test_s_open_error
TestGemInstaller#test_generate_bin_bindir_with_user_install_warning
TestRDocMarkupPreProcess#test_include_file
TestRDocMarkupPreProcess#test_include_file_encoding_incompatible
TestRubyOptions#test_encoding
TestUnicodeEscape#test_basic
TestWEBrickCGI#test_cgi
TestWin32OLE#test_s_codepage_changed


Versus your patch:
TestDir_M17N#test_filename_bytes_euc_jp
TestDir_M17N#test_filename_euc_jp
TestDir_M17N#test_filename_ext_euc_jp_and_int_utf_8
TestDir_M17N#test_filename_extutf8
TestDir_M17N#test_filename_extutf8_inteucjp_representable
TestDir_M17N#test_filename_extutf8_inteucjp_unrepresentable
TestDir_M17N#test_filename_utf8_raw_name
TestGDBM#test_reorganize
TestGDBM#test_s_open_create_new
TestGDBM#test_s_open_error
TestGemInstaller#test_generate_bin_bindir_with_user_install_warning
TestOpen3#test_capture3_flip
TestRDocMarkupPreProcess#test_include_file
TestRDocMarkupPreProcess#test_include_file_encoding_incompatible
TestRubyOptions#test_encoding
TestUnicodeEscape#test_basic
TestWEBrickCGI#test_cgi
TestWin32OLE#test_s_codepage_changed

===

In your patch TestArgf#test_textmode no longer fails, but TestOpen3#test_capture3_flip does.

trunk:

10347 tests, 1884751 assertions, 9 failures, 9 errors, 95 skips

your patch:

10347 tests, 1884748 assertions, 8 failures, 10 errors, 95 skips

===

It is worth to mention that write performance using your test script did improve:

V:\>ruby -v t.rb
ruby 1.8.7 (2011-06-30 patchlevel 352) [i386-mingw32]
"0 0.598029"
"1 0.300001"
"2 0.2"

ruby 1.9.2p290 (2011-07-09) [i386-mingw32]
"0 0.413023"
"1 0.746043"
"2 0.935052"

ruby 1.9.3p0 (2011-10-30) [i386-mingw32]
"0 0.400014"
"1 0.721023"
"2 0.918049"

V:\>ruby -v t.rb
ruby 2.0.0dev (2011-11-10 trunk 33700) [i386-mingw32]
"0 0.396023"
"1 0.725817"
"2 0.936044"

*with patch*
V:\>ruby -v t.rb
ruby 2.0.0dev (2011-11-10 trunk 33700) [i386-mingw32]
"0 0.401023"
"1 0.238013"
"2 0.139008"

I'm assigning this to Usaku Nakamura for comments.

Thank you.
----------------------------------------
Feature #5562: Improvement of Windows IO performance
http://redmine.ruby-lang.org/issues/5562

Author: Hiroshi Shirosaki
Status: Open
Priority: Normal
Assignee: Usaku NAKAMURA
Category: core
Target version: 


=begin
I suggest a patch to improve Windows IO performance.

Ruby's text mode IO is much slower than binary mode.
On Windows text mode is default, so Windows IO is slow.
I assume that's mainly because of CRLF linefeed code conversion.

My idea to improve IO performance is as below.
- Change default linefeed conversion from Universal newline to CRLF newline on Windows
- Use binary mode process with OS's text mode if only CRLF conversion is needed
- Use Ruby's text mode with universal newline conversion if encoding conversion is needed

Although that causes io.c code to be more complicated, IO with CRLF conversion performance seems to be improved much.
I confirmed "make test-all TEST=ruby" have been passed. There was 3 errors, but ruby without this patch had same errors.
I think this patch doesn't affect other OS.

Line endings of "p" or "puts" writing is LF on trunk, but CRLF on 1.8.7 or 1.9.2.
This patch reverts to CRLF.


Here is #1332 benchmark test and results.

time = [Time.new]
c = ''
'aaaa'.upto('zzzz') {|e| c << e}
4.times { c << c }
time << Time.new
File.open('out.file','w') { |f| f.write(c) }
time << Time.new
c = File.open('out.file','r') { |f| f.read }
time << Time.new
0.upto(time.size - 2) {|i| p "#{i} #{time[i+1]-time[i]}" }


- Result

ruby 1.8.7 (2011-06-30 patchlevel 352) [i386-mingw32]
"0 0.78125"
"1 0.6875"
"2 0.5625"

ruby 2.0.0dev (2011-11-03) [i386-mingw32]
"0 0.59375"
"1 1.09375"
"2 1.296875"

ruby 2.0.0dev (2011-11-03 trunk 33615) [i386-mingw32] with this patch
"0 0.625"
"1 0.65625"
"2 0.34375"
=end



-- 
http://redmine.ruby-lang.org