[ruby-core:80269] [Ruby trunk Bug#13349] Ruby unable to open files in /dev on AIX if compiled on AIX 7.1 or later

Issue #13349 has been updated by 0x6373 (Christian Schwabl).

Description updated

Clarified that the supplied patch applies to ruby-2.0.0p468

----------------------------------------
Bug #13349: Ruby unable to open files in /dev on AIX if compiled on AIX 7.1 or later
https://bugs.ruby-lang.org/issues/13349#change-63706

* Author: 0x6373 (Christian Schwabl)
* Status: Open
* Priority: Normal
* Assignee: 
* Target version: 
* ruby -v: 
* Backport: 2.2: UNKNOWN, 2.3: UNKNOWN, 2.4: UNKNOWN
----------------------------------------
In io.c there are 2 Linux specific flags (O_CLOEXEC and O_NOINHERIT) which are set regardless of the operating system it is compiled on.

Since AIX 7.1 /usr/include/fcntl.h defines O_CLOEXEC with the following statement:

~~~ c
#if _XOPEN_SOURCE >= 700                                                                       
/* Currently as all 32 bits were used by _F flags, O_CLOEXEC and O_NOFOLLOW was defined as 64 bit.                                                                                             
 * Open function takes int as parameter hence we can't use 64bit O_CLOEXEC and O_NOFOLLOW for open.                                                                                            
 * instead of changing _FCLOEXEC and _FNOFOLLOW to 32 bit we have decided to use _FDEFERIND to define                                                                                          
 * O_CLOEXEC and _FCLREAD to define O_NOFOLLOW as there is no corresponding O_ flag for _FDEFERIND and _FCLREAD.                                                                               
 */                                                                                            
#define O_CLOEXEC       _FDEFERIND      /* sets FD_CLOEXEC on open      */                     
#define O_NOFOLLOW      _FCLREAD        /* do not follow symlinks       */                     

/* In AIX TTY kernel extension, The terminal parameters are automatically                      
 * set to default values on first open. Hence O_TTY_INIT will be defined as zero.              
 */                                                                                            
#define O_TTY_INIT      0               /* sets conforming tty state    */                     
#endif /* _XOPEN_SOURCE >= 700 */                                                              
~~~

So, on AIX flag _FDEFERIND gets set, which causes all file open operations to /dev (i.e. /dev/null )to fail with EINVAL.

The following patch encloses the linux-only part with "#ifdev __linux__"

~~~ diff
--- a/io.c        2017-03-20 14:23:58.117976095 +0100                                                  
+++ b/io.c        2017-03-21 14:28:08.900827149 +0100                                                  
@@ -209,11 +209,13 @@ int                         
 rb_cloexec_open(const char *pathname, int flags, mode_t mode)                                       
 {                                                
     int ret;                                     
-#ifdef O_CLOEXEC                                 
+#ifdef __linux__                                 
+  #ifdef O_CLOEXEC                               
     /* O_CLOEXEC is available since Linux 2.6.23.  Linux 2.6.18 silently ignore it. */              
     flags |= O_CLOEXEC;                          
-#elif defined O_NOINHERIT                        
+  #elif defined O_NOINHERIT                      
     flags |= O_NOINHERIT;                        
+  #endif                                         
 #endif                                           
     ret = open(pathname, flags, mode);           
     if (ret == -1) return -1;                    
~~~

This patch applies against ruby-2.0.0p468 since i had a working reference of this compiled on AIX 5.3.

I tried this with ruby-2.0.0p468, ruby-2.2.6 and 2.3.x, but trunk is also affected since this part of the code hasn't changed in a while (end of 2011 from what i found).

Is there any style-guide for this part of the code?





-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>