[ruby-core:98842] [Ruby master Bug#16965] io.c: Unsupported copy_file_range() not detected properly on certain kernels

Issue #16965 has been updated by mame (Yusuke Endoh).

Assignee set to Glass_saga (Masaki Matsushita)

I think that Ruby handles the ENOSYS appropriately and fall back to fcopyfile and sendfile: https://github.com/ruby/ruby/blob/41a4c80d284a24360d3a6c5b6e5ca408ccca6efc/io.c#L11084

"Operation not supported" error is not ENOSYS but EOPNOTSUP, which Ruby does not handle. The issue is that the recent RedHat kernel may return EOPNOTSUP.  The RedHat developers are aware of the incompatibility, and looks like they will change the return value to ENOSYS: https://bugzilla.redhat.com/show_bug.cgi?id=1783554

So, please wait for their fix.

Aside from that, is it good for Ruby to rescue EOPNOTSUP for future robustness?  @glass_saga

----------------------------------------
Bug #16965: io.c: Unsupported copy_file_range() not detected properly on certain kernels
https://bugs.ruby-lang.org/issues/16965#change-86204

* Author: stanhu (Stan Hu)
* Status: Open
* Priority: Normal
* Assignee: Glass_saga (Masaki Matsushita)
* ruby -v: ruby 2.6.6p146 (2020-03-31 revision 67876) [x86_64-linux]
* Backport: 2.5: UNKNOWN, 2.6: UNKNOWN, 2.7: UNKNOWN
----------------------------------------
In recent RedHat kernels (for example: RHEL 7.8 using kernel 3.10.0-1127.8.2.el7.x86_64), `copy_file_range()` has been disabled and now returns `ENOSYS` when Ruby attempts to call this in `IO.copy_stream`. A simple `FileUtils.copy_file` will fail with `Operation not supported - copy_file_range` on these kernels. 

[This sample program](https://gitlab.com/kevenhughes/arcport/-/blob/master/cfr.c) demonstrates how to see the `ENOSYS` error:

```
$ strace ./a.out 
execve("./a.out", ["./a.out"], 0x7ffde8ca0680 /* 38 vars */) = 0
brk(NULL)                               = 0x24bc000
brk(0x24bd1c0)                          = 0x24bd1c0
arch_prctl(ARCH_SET_FS, 0x24bc880)      = 0
uname({sysname="Linux", nodename="xxx", ...}) = 0
readlink("/proc/self/exe", "/home/xxx/tmp/a.out", 4096) = 21
brk(0x24de1c0)                          = 0x24de1c0
brk(0x24df000)                          = 0x24df000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "./1", O_RDONLY)       = 3
openat(AT_FDCWD, "./2", O_WRONLY)       = 4
copy_file_range(3, NULL, 4, NULL, 1, 0) = -1 ENOSYS (Function not implemented)
fstat(3, {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
fstat(4, {st_mode=S_IFREG|0664, st_size=0, ...}) = 0
fcntl(4, F_GETFL)                       = 0x8001 (flags O_WRONLY|O_LARGEFILE)
read(3, "", 1)                          = 0
exit_group(0)                           = ?
+++ exited with 0 +++
```

This was possibly changed during a recent security release: https://access.redhat.com/errata/RHSA-2020:1465

Ruby's io.c detects whether `copy_file_range()` is defined, not whether it is actually supported. The following test program illustrates the hole in the detection mechanism:


```
#include <syscall.h>
#include <stdio.h>

#if defined __linux__ && defined __NR_copy_file_range
#  define USE_COPY_FILE_RANGE 1
#else
#  define USE_COPY_FILE_RANGE 0
#endif

int main()
{
  printf("copy_file_range? %d\n", USE_COPY_FILE_RANGE);
}
```

`USE_COPY_FILE_RANGE` gets set to 1 even in when the system call doesn't succeed. 

I suggest a few improvements:

1. Use a compile-time test to verify that `copy_file_range()` can actually be executed. 
2. Make it possible to disable `USE_COPY_FILE_RANGE` via a build option. Since the test in 1 could still pass if it is run on a Docker host that supports `copy_file_range()`, it would be helpful for us to manually disable it.

Reported by GitLab customers: https://gitlab.com/gitlab-org/gitlab/-/issues/218999




-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>