[ruby-core:55103] [ruby-trunk - Bug #8431] File.read() crash on Win32SP3 32bit

Issue #8431 has been updated by phasis68 (Heesob Park).


This bug is due to the invalid error checking of SetFilePointer function.
The constant INVALID_SET_FILE_POINTER is defined as (DWORD)-1 and is same to 0xFFFFFFFF(4294967295).

I can see the following sentences in the documentation of SetFilePointer function:
Because INVALID_SET_FILE_POINTER is a valid value for the low-order DWORD of the new file pointer, you must check both the return value of the function and the error code returned by GetLastError to determine whether or not an error has occurred. If an error has occurred, the return value of SetFilePointer is INVALID_SET_FILE_POINTER and GetLastError returns a value other than NO_ERROR.

Refer to http://msdn.microsoft.com/en-us/library/windows/desktop/aa365541(v=vs.85).aspx

Here is a patch:
diff --git a/win32.c b/win32.c.new
index 318af2f..18bd378 100644
--- a/win32.c
+++ b/win32.c.new
@@ -6084,7 +6084,7 @@ rb_w32_read(int fd, void *buf, size_t size)
 #ifndef INVALID_SET_FILE_POINTER
 #define INVALID_SET_FILE_POINTER ((DWORD)-1)
 #endif
-	    if (low == INVALID_SET_FILE_POINTER) {
+	    if (low == INVALID_SET_FILE_POINTER && GetLastError() != NO_ERROR) {
 		errno = map_errno(GetLastError());
 		MTHREAD_ONLY(LeaveCriticalSection(&_pioinfo(fd)->lock));
 		return -1; 

----------------------------------------
Bug #8431: File.read() crash on Win32SP3 32bit
https://bugs.ruby-lang.org/issues/8431#change-39469

Author: galnaktar (Oleg K)
Status: Open
Priority: Normal
Assignee: cruby-windows
Category: platform/windows
Target version: 
ruby -v: ruby 2.0.0p195 (2013-05-14) [i386-mingw32]
Backport: 1.9.3: UNKNOWN, 2.0.0: UNKNOWN


Open existing empty file, seek on 0xFFFFFFFF(4294967295) and trying to read 1 byte.


c:\Ruby200\bin>irb
DL is deprecated, please use Fiddle
irb(main):001:0> f = File.open("1", "w")
=> #<File:1>
irb(main):002:0> f.close
=> nil
irb(main):003:0> f = File.open("1", "rb")
=> #<File:1>
irb(main):004:0> f.seek(4294967295)
=> 0
irb(main):005:0> f.read(1)
(irb):5: [BUG] rb_sys_fail_str(1) - errno == 0
ruby 2.0.0p195 (2013-05-14) [i386-mingw32]

-- Control frame information -----------------------------------------------
c:0019 p:---- s:0076 e:000075 CFUNC  :read
c:0018 p:0007 s:0072 e:000071 EVAL   (irb):5 [FINISH]
c:0017 p:---- s:0070 e:000069 CFUNC  :eval
c:0016 p:0024 s:0063 e:000062 METHOD c:/Ruby200/lib/ruby/2.0.0/irb/workspace.rb:


-- 
http://bugs.ruby-lang.org/