From: "peterzhu2118 (Peter Zhu) via ruby-core" <ruby-core@...>
Date: 2024-10-30T19:43:25+00:00
Subject: [ruby-core:119642] [Ruby master Bug#20853] Hash key retrieval after Process.warmup

Issue #20853 has been updated by peterzhu2118 (Peter Zhu).


Thank you for reporting this bug.

I believe the issue here is that because `Process.warmup` runs a GC compaction, it can cause objects to move. While this is usually not a problem, Proc#hash is calculated using pointer values. Since the pointer values can change, the hash value can change as well. For example:

```ruby
p = proc {}

puts p.hash # => 2411193347801829553

GC.verify_compaction_references(expand_heap: true, toward: :empty)

puts p.hash # => 1198623829435689723
```

This causes the hash value for the Hash to change as well:

```ruby
hash = {}
key = {:namespace=>"features", :priority=>1, :get_dependency_keys=>proc {} }

hash[key] = 1

puts hash.key?(key) # => true

GC.verify_compaction_references(expand_heap: true, toward: :empty)

puts hash.key?(key) # => false
```

I'll work on a fix for this problem.

----------------------------------------
Bug #20853: Hash key retrieval after Process.warmup
https://bugs.ruby-lang.org/issues/20853#change-110292

* Author: keelerm84 (Matthew Keeler)
* Status: Open
* ruby -v: ruby 3.3.5 (2024-09-03 revision ef084cc8f4) [x86_64-linux]
* Backport: 3.1: UNKNOWN, 3.2: UNKNOWN, 3.3: UNKNOWN
----------------------------------------
This was first reported as an issue against the [LaunchDarkly SDK](https://github.com/launchdarkly/ruby-server-sdk/issues/282) and [Sidekiq SDKs](https://github.com/sidekiq/sidekiq/issues/6279) as an issue in Ruby 3.3.1. I can still in Ruby 3.3.5.

**Overview of behavior**
The LaunchDarkly SDK maintains a hash of feature and segment information. This hash is indexed by 2 keys which are themselves hashes. They are defined as:

```
  FEATURES = {
    namespace: "features",
    priority: 1,
    get_dependency_keys: lambda { |flag| (flag[:prerequisites] || []).map { |p| p[:key] } },
  }.freeze

  SEGMENTS = {
    namespace: "segments",
    priority: 0,
  }.freeze
```

When running the attached script, we often see an error indicating that the flag is not found. If you comment out the `Process.warmup` line, this error will go away.

The `require 'active_support/all'` directive is not strictly necessary. Including it causes the failure to more frequently occur, presumably due to the increase work in the warmup phase.

**Hash key and access**
When debugging the LaunchDarkly code, I noticed some odd behavior. Even though the hash seems populated, the FEATURES key could not index into it correctly.

Below is a bit of code, modified from the SDK, to show how we are trying to interact with the hash. The puts are added to clarify the issue.

```ruby
    def get(kind, key)
      @lock.with_read_lock do

        @items.keys.each do |k|
          puts "##### Does #{kind[:namespace]} match the key #{k}? #{k == kind} #{k.eql?(kind)} #{k.hash == kind.hash}"
        end
        puts "###### Does @items include it? #{@items.include?(kind)}"

        coll = @items[kind]
        f = coll.nil? ? nil : coll[key.to_sym]
        (f.nil? || f[:deleted]) ? nil : f
      end
    end
```

*Running without Process.warmup*

If we run the attached script without the `Process.warmup` call, we will see that the keys align with the FEATURES key we provided. Additionally, the hash does see that the key exists, resulting in an evaluation result of true at the end.

That output follows.


```shell
##### Does features match the key {:namespace=>"segments", :priority=>0}? false false false
##### Does features match the key {:namespace=>"features", :priority=>1, :get_dependency_keys=>#<Proc:0x00007ee6174f9738 /home/mkeeler/code/launchdarkly/ruby-server-sdk.git/main/lib/ldclient-rb/in_memory_store.rb:17 (lambda)>}? true true true
###### Does @items include it? true
true
```

*Running with Process.warmup*

If you enable the `Process.warmup` option, and run the script a few times, you will see output similar to the following.

Note that while the keys are `==`, `eql?`, and their `.hash` values are equal, the hash does not see that the FEATURES key exists.


```shell
##### Does features match the key {:namespace=>"segments", :priority=>0}? false false false
##### Does features match the key {:namespace=>"features", :priority=>1, :get_dependency_keys=>#<Proc:0x0000774f1cbfd730 /home/mkeeler/code/launchdarkly/ruby-server-sdk.git/main/lib/ldclient-rb/in_memory_store.rb:17 (lambda)>}? true true true
###### Does @items include it? false
I, [2024-10-30T14:34:33.842145 #974568]  INFO -- : [LDClient] Unknown feature flag "sample-feature". Returning default value
false
```


**Workarounds**

We have worked around this issue by replacing these keys with classes. You can refer to the [PR here](https://github.com/launchdarkly/ruby-server-sdk/pull/301/files) for those details.

You can also resolve this by triggering a `rehash` on the hash after the `Process.warmup` has run.

---Files--------------------------------
main.rb (630 Bytes)
Gemfile (96 Bytes)


-- 
https://bugs.ruby-lang.org/
 ______________________________________________
 ruby-core mailing list -- ruby-core@ml.ruby-lang.org
 To unsubscribe send an email to ruby-core-leave@ml.ruby-lang.org
 ruby-core info -- https://ml.ruby-lang.org/mailman3/lists/ruby-core.ml.ruby-lang.org/