From: Eric Wong <normalperson@...>
Date: 2014-01-15T18:37:00+00:00
Subject: [ruby-core:59790] Re: [ruby-trunk - Bug #5985] miniruby skews "make benchmark" results

eregontp@gmail.com wrote:
> Eric Wong wrote:
> > ko1 (Koichi Sasada) wrote:
> >  > (or remove --disable-gems ?)
> >  
> >  I favor removing --disable-gems from the benchmark command.
> >  I think it's unrealistic to use/benchmark Ruby without gems nowadays.
> 
> Unfortunately that would make the comparison between different ruby
> versions more brittle as the gems might change, particularly how much
> allocations they do, no?

Good point.  Perhaps keeping --disable-gems globally is better.

> I have found as well some benchmarks in benchmark/ are very sensitive
> to the heap usage.

Exactly.