From: "byroot (Jean Boussier) via ruby-core" <ruby-core@...>
Date: 2023-09-20T13:01:12+00:00
Subject: [ruby-core:114838] [Ruby master Bug#19624] Backticks - IO object leakage

Issue #19624 has been updated by byroot (Jean Boussier).





> Do you have any short reproducible code?



No sorry, I don't understand the fiber scheduler enough to reduce the test suite.



I discovered the bug via: https://github.com/rmosolgo/graphql-ruby/issues/4640#issuecomment-1727220148



```

GraphQL::Dataloader::AsyncDataloader::With the toy scheduler from Ruby's tests#test_0003_works with GraphQL:

NotImplementedError: method `hash' called on hidden T_FILE object (0x0000000119cd5ca0 flags=0xb)

    src/graphql-ruby/spec/support/dummy_scheduler.rb:159:in `io_wait'

    src/graphql-ruby/spec/graphql/dataloader/async_dataloader_spec.rb:69:in ``'

    src/graphql-ruby/spec/graphql/dataloader/async_dataloader_spec.rb:69:in `sleep'

```



I tried extracting their scheduler to make a self contained repro, but without success:



```ruby

class DummyScheduler

  def initialize

    @readable = {}

    @writable = {}

    @waiting = {}



    @closed = false



    @lock = Mutex.new

    @blocking = 0

    @ready = []



    @urgent = IO.pipe

  end



  attr :readable

  attr :writable

  attr :waiting



  def next_timeout

    _fiber, timeout = @waiting.min_by{|key, value| value}



    if timeout

      offset = timeout - current_time



      if offset < 0

        return 0

      else

        return offset

      end

    end

  end



  def run

    # $stderr.puts [__method__, Fiber.current].inspect



    while @readable.any? or @writable.any? or @waiting.any? or @blocking.positive?

      # Can only handle file descriptors up to 1024...

      readable, writable = IO.select(@readable.keys + [@urgent.first], @writable.keys, [], next_timeout)



      # puts "readable: #{readable}" if readable&.any?

      # puts "writable: #{writable}" if writable&.any?



      selected = {}



      readable && readable.each do |io|

        if fiber = @readable.delete(io)

          selected[fiber] = IO::READABLE

        elsif io == @urgent.first

          @urgent.first.read_nonblock(1024)

        end

      end



      writable && writable.each do |io|

        if fiber = @writable.delete(io)

          selected[fiber] |= IO::WRITABLE

        end

      end



      selected.each do |fiber, events|

        fiber.resume(events)

      end



      if @waiting.any?

        time = current_time

        waiting, @waiting = @waiting, {}



        waiting.each do |fiber, timeout|

          if fiber.alive?

            if timeout <= time

              fiber.resume

            else

              @waiting[fiber] = timeout

            end

          end

        end

      end



      if @ready.any?

        ready = nil



        @lock.synchronize do

          ready, @ready = @ready, []

        end



        ready.each do |fiber|

          fiber.resume

        end

      end

    end

  end



  def close

    # $stderr.puts [__method__, Fiber.current].inspect



    raise "Scheduler already closed!" if @closed



    self.run

  ensure

    @urgent.each(&:close)

    @urgent = nil



    @closed = true



    # We freeze to detect any unintended modifications after the scheduler is closed:

    self.freeze

  end



  def closed?

    @closed

  end



  def current_time

    Process.clock_gettime(Process::CLOCK_MONOTONIC)

  end



  def timeout_after(duration, klass, message, &block)

    fiber = Fiber.current



    self.fiber do

      sleep(duration)



      if fiber && fiber.alive?

        fiber.raise(klass, message)

      end

    end



    begin

      yield(duration)

    ensure

      fiber = nil

    end

  end



  def process_wait(pid, flags)

    # $stderr.puts [__method__, pid, flags, Fiber.current].inspect



    # This is a very simple way to implement a non-blocking wait:

    Thread.new do

      Process::Status.wait(pid, flags)

    end.value

  end



  def io_wait(io, events, duration)

    p io

    # $stderr.puts [__method__, io, events, duration, Fiber.current].inspect



    unless (events & IO::READABLE).zero?

      @readable[io] = Fiber.current

    end



    unless (events & IO::WRITABLE).zero?

      @writable[io] = Fiber.current

    end



    Fiber.yield

  end



  # Used for Kernel#sleep and Mutex#sleep

  def kernel_sleep(duration = nil)

    # $stderr.puts [__method__, duration, Fiber.current].inspect



    self.block(:sleep, duration)



    return true

  end



  # Used when blocking on synchronization (Mutex#lock, Queue#pop, SizedQueue#push, ...)

  def block(blocker, timeout = nil)

    # $stderr.puts [__method__, blocker, timeout].inspect



    if timeout

      @waiting[Fiber.current] = current_time + timeout

      begin

        Fiber.yield

      ensure

        # Remove from @waiting in the case #unblock was called before the timeout expired:

        @waiting.delete(Fiber.current)

      end

    else

      @blocking += 1

      begin

        Fiber.yield

      ensure

        @blocking -= 1

      end

    end

  end



  # Used when synchronization wakes up a previously-blocked fiber (Mutex#unlock, Queue#push, ...).

  # This might be called from another thread.

  def unblock(blocker, fiber)

    # $stderr.puts [__method__, blocker, fiber].inspect

    # $stderr.puts blocker.backtrace.inspect

    # $stderr.puts fiber.backtrace.inspect



    @lock.synchronize do

      @ready << fiber

    end



    io = @urgent.last

    io.write_nonblock('.')

  end



  def fiber(&block)

    fiber = Fiber.new(blocking: false, &block)



    fiber.resume



    return fiber

  end



  def address_resolve(hostname)

    Thread.new do

      Addrinfo.getaddrinfo(hostname, nil).map(&:ip_address).uniq

    end.value

  end

end



Fiber.set_scheduler(DummyScheduler.new)



p `sleep 2`

```



Perhaps @ioquatix would know how to reproduce?





----------------------------------------

Bug #19624: Backticks - IO object leakage

https://bugs.ruby-lang.org/issues/19624#change-104683



* Author: pineman (Jo�o Pinheiro)

* Status: Open

* Priority: Normal

* ruby -v: ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin22]

* Backport: 3.0: UNKNOWN, 3.1: UNKNOWN, 3.2: UNKNOWN

----------------------------------------

Hi,



This code works on ruby 3.0.6:



```ruby

`echo`

ObjectSpace.each_object(IO) do |io|

  if ![STDIN, STDOUT, STDERR].include?(io)

    io.close

  end

end

```

but raises `IOError` on 3.2.2:

```

minimal-repro-case.rb:8:in `close': uninitialized stream (IOError)

```





I found it started failing on ruby 3.1.0 and after, on macOS and Linux.

This code is useful for closing unneeded IO objects in forked processes.

It looks like backticks is 'leaking' IO objects, waiting for GC, and it didn't used to before 3.1.0.

In ruby 3.1.0, inside `rb_f_backquote` in `io.c`, `rb_gc_force_recycle` was removed in favor of `RB_GC_GUARD` (commit `aeae6e2842e`). I wonder if this has something to do with the problem.

Is this code incorrect since ruby 3.1.0 or is it a bug in ruby?

Thanks.



---Files--------------------------------

minimal-repro-case.rb (109 Bytes)





-- 

https://bugs.ruby-lang.org/

 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/postorius/lists/ruby-core.ml.ruby-lang.org/