[ruby-core:108790] [Ruby master Bug#18811] PTY I/O not working on AIX 7.x

Issue #18811 has been updated by hspem (Per-Erik Martin).

File pty.c.patch added

mame (Yusuke Endoh) wrote in #note-2:
> 
> This is my impression, but the maintenance state of AIX is not good. The CI for AIX has not been working for a long time. This issue is probably unlikely to be fixed unless you provide a patch; even if a patch is created, we cannot promise that we will apply it.

I found a way to fix this. The problem seems to be a known strangeness in the AIX pty implementation. It's sensitive to in which order things are done relative closing the descriptors. It's more common that one of the processes simply hangs if you get it wrong, but apparently you can also lose output. We have had a similar problem in sshd where it would hang unless you synchronized the parent and child processes and made the child wait for the master to close the slave end before proceeding. (And I have had processes hanging in ruby's pty as well, although not as often.) So the issue seems to be when the parent process does "close(slave)" and when this happens in relation to what the child process is doing.

Adding some kind of synchronization isn't easy in ext/pty/pty.c, but there's another way:
Since AIX does not have TIOCSTTY defined, chfunc() will close "slave" and reopen the device. This means that the parent process can close its slave descriptor *before* forking, instead of afterwards. This resolves the race condition.

The patch is simple, attached.



----------------------------------------
Bug #18811: PTY I/O not working on AIX 7.x
https://bugs.ruby-lang.org/issues/18811#change-97861

* Author: hspem (Per-Erik Martin)
* Status: Feedback
* Priority: Normal
* Assignee: kanemoto (Yutaka Kanemoto)
* ruby -v: ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [powerpc-aix7.1.0]
* Backport: 2.7: UNKNOWN, 3.0: UNKNOWN, 3.1: UNKNOWN
----------------------------------------
The attached test script simply executes a command under a PTY and captures the output and exit code.
This works on Linux (all supported versions of Redhat, Debian, Ubuntu, and SuSE) as well as Solaris 11.4 on sparc and x86.
But on AIX 7.1 - 7.3, it doesn't. This was tested with a ruby 3.1.2 that I built, but it exhibits the same behavior on other ruby versions as well, like 2.7.5 from IBM's freeware repo.

The core of the attached program is:
``` ruby
begin
  PTY.spawn($command) do |r, w, pid|
    w.close
    begin
      r.each do |line|
        puts "Got line \"#{line.chomp}\""
      end
    rescue Errno::EIO
      # ignore
    rescue => e
      $stderr.puts "Read error: #{e}"
    ensure
      Process.wait pid
    end
    $exit_status = $?.exitstatus
  end
rescue => e
  $stderr.puts "PTY.spawn error: #{e}"
end
```


This should read the output from echo but doesn't:
``` shell
# ./ptytest.rb 'echo b'
Exit status 0
# 
```
It does execute the command, but it seems it can't read the output:
``` shell
# ./ptytest.rb 'exit 3'
Exit status 3
# cat /tmp/e.txt
cat: 0652-050 Cannot open /tmp/e.txt.
# ./ptytest.rb 'echo b > /tmp/e.txt'
Exit status 0
# cat /tmp/e.txt
b
# 
```
However, the unit tests for PTY work, so how it this possible? It turns out that all the unit tests run ruby (tinyruby really), and this works:
``` shell
# ./ptytest.rb 'ruby -e "puts \"b\""'
Got line "b"
Exit status 0
# 
```
as well as this:
``` shell
# ./ptytest.rb 'ruby -e "system \"echo b\""'
Got line "b"
Exit status 0
# 
```

So I'm at a loss here. How come it works to use "ruby" but no other commands? Is there something wrong with the script? If so, why does it work on all other platforms?


---Files--------------------------------
ptytest.rb (534 Bytes)
pty.c.patch (655 Bytes)


-- 
https://bugs.ruby-lang.org/

Unsubscribe: <mailto:[e-mail]?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-core>