[ruby-core:121238] [Ruby master Misc#19122] Use MADV_DONTNEED instead of MADV_FREE when freeing a Fiber's stack

Issue #19122 has been updated by ioquatix (Samuel Williams).


On the latest Linux kernel, the constants have changed:

```
MADV_DONTNEED = 4
MADV_FREE = 8
```

I did a quick comparison, creating 10,000 fibers x10 times in a loop:

```
samuel@aiko ~/D/s/memory-leak (main)> time RUBY_SHARED_FIBER_POOL_FREE_STACKS=8 bundle exec test.rb >/dev/null
<main>: warning: Setting RUBY_SHARED_FIBER_POOL_FREE_STACKS to a value greater than 1 is operating system specific, and may cause crashes.

________________________________________________________
Executed in  841.07 millis    fish           external
   usr time  373.88 millis  441.00 micros  373.44 millis
   sys time  463.22 millis  168.00 micros  463.05 millis

samuel@aiko ~/D/s/memory-leak (main)> time RUBY_SHARED_FIBER_POOL_FREE_STACKS=16 bundle exec test.rb >/dev/null
<main>: warning: Setting RUBY_SHARED_FIBER_POOL_FREE_STACKS to a value greater than 1 is operating system specific, and may cause crashes.

________________________________________________________
Executed in  582.77 millis    fish           external
   usr time  357.66 millis  420.00 micros  357.24 millis
   sys time  220.90 millis  164.00 micros  220.74 millis

samuel@aiko ~/D/s/memory-leak (main)> time RUBY_SHARED_FIBER_POOL_FREE_STACKS=0 bundle exec test.rb >/dev/null

________________________________________________________
Executed in  444.61 millis    fish           external
   usr time  312.92 millis  503.00 micros  312.41 millis
   sys time  128.99 millis    0.00 micros  128.99 millis
```

Also memory usage as we expected:

```
samuel@aiko ~/D/s/memory-leak (main)> RUBY_SHARED_FIBER_POOL_FREE_STACKS=16 bundle exec test.rb
<main>: warning: Setting RUBY_SHARED_FIBER_POOL_FREE_STACKS to a value greater than 1 is operating system specific, and may cause crashes.
PID: 70235
Memory usage: 150.82 MB
Clearing fibers and garbage collecting...
Memory usage: 150.82 MB
samuel@aiko ~/D/s/memory-leak (main)> RUBY_SHARED_FIBER_POOL_FREE_STACKS=8 bundle exec test.rb
<main>: warning: Setting RUBY_SHARED_FIBER_POOL_FREE_STACKS to a value greater than 1 is operating system specific, and may cause crashes.
PID: 70279
Memory usage: 151.63 MB
Clearing fibers and garbage collecting...
Memory usage: 73.63 MB
```

Using the following test script:

```
#!/usr/bin/env ruby

require "memory/leak/system"

def print_memory_usage
  size = Memory::Leak::System.memory_usage(Process.pid)
  units = %w(B KB MB GB TB)
  unit = 0
  
  while size > 1024 && unit < units.size
    size /= 1024.0
    unit += 1
  end
  
  puts "Memory usage: %.2f %s" % [size, units[unit]]
end

puts "PID: #{Process.pid}"

10.times do |i|
  puts "Iteration: #{i}"
  
  # Create 10,000 fibers:
  fibers = 10_000.times.map do
    Fiber.new do
      Fiber.yield Fiber.current
    end.resume
  end

  # Print memory usage:
  print_memory_usage

  puts "Clearing fibers and garbage collecting..."

  # Clear fibers:
  fibers.clear
  GC.start

  # Print memory usage:
  print_memory_usage
end
```

----------------------------------------
Misc #19122: Use MADV_DONTNEED instead of MADV_FREE when freeing a Fiber's stack
https://bugs.ruby-lang.org/issues/19122#change-112186

* Author: smcgivern (Sean McGivern)
* Status: Assigned
* Assignee: ioquatix (Samuel Williams)
----------------------------------------
I'd like to propose that Ruby stops using MADV_FREE when freeing a Fiber's stack, and switches to using MADV_DONTNEED even when MADV_FREE is supported.

MADV_FREE is used in one place in the Ruby codebase, when freeing the stack of a freed Fiber: https://git.ruby-lang.org/ruby.git/tree/cont.c#n683

The comment for `fiber_pool_stack_free` says:

```c
// We advise the operating system that the stack memory pages are no longer being used.
// This introduce some performance overhead but allows system to relaim memory when there is pressure.
```

Where possible (i.e. on Linux 4.5 and later), `fiber_pool_stack_free` uses `MADV_FREE` over `MADV_DONTNEED`. This has the side effect that memory statistics such as RSS will not reduce until and unless the OS actually reclaims that memory. If that doesn't happen, then the reported memory usage via RSS will be much higher than the 'real' memory usage.

If this was pervasive throughtout the Ruby codebase then that would be one thing, but currently this is just for Fiber. This means that:

1. A program that doesn't use Fiber will have somewhat reliable RSS statistics on recent Linux.
2. A program that heavily uses Fiber (such as something using Async::HTTP) will see an inflated RSS statistic.

Go made a similar change to the one I'm proposing here for similar reasons: https://github.com/golang/go/issues/42330

> While `MADV_FREE` is somewhat faster than `MADV_DONTNEED`, it doesn't affect many of the statistics that `MADV_DONTNEED` does until the memory is actually reclaimed. This generally leads to poor user experience, like confusing stats in `top` and other monitoring tools; and bad integration with management systems that respond to memory usage.
> [...]
> I propose we change the default to prefer `MADV_DONTNEED` over `MADV_FREE`, to favor user-friendliness and minimal surprise over performance. I think it's become clear that Linux's implementation of `MADV_FREE` ultimately doesn't meet our needs.

As an aside, MADV_FREE was not used in Ruby 3.1 (https://bugs.ruby-lang.org/issues/19101), and I haven't found any bugs filed about this behaviour other than that one.



-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- [e-mail]
 To unsubscribe send an email to [e-mail]
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/