[#13161] hacking on the "heap" implementation in gc.c — Lloyd Hilaiel <lloyd@...>

Hi all,

16 messages 2007/11/01

[#13182] Thinking of dropping YAML from 1.8 — Urabe Shyouhei <shyouhei@...>

Hello all.

14 messages 2007/11/03

[#13315] primary encoding and source encoding — David Flanagan <david@...>

I've got a couple of questions about the handling of primary encoding.

29 messages 2007/11/08
[#13331] Re: primary encoding and source encoding — Yukihiro Matsumoto <matz@...> 2007/11/09

Hi,

[#13368] method names in 1.9 — "David A. Black" <dblack@...>

Hi --

61 messages 2007/11/10
[#13369] Re: method names in 1.9 — Yukihiro Matsumoto <matz@...> 2007/11/10

Hi,

[#13388] Re: method names in 1.9 — Charles Oliver Nutter <charles.nutter@...> 2007/11/11

Yukihiro Matsumoto wrote:

[#13403] Re: method names in 1.9 — "Austin Ziegler" <halostatue@...> 2007/11/11

On 11/11/07, Charles Oliver Nutter <charles.nutter@sun.com> wrote:

[#13410] Re: method names in 1.9 — David Flanagan <david@...> 2007/11/11

Austin Ziegler wrote:

[#13413] Re: method names in 1.9 — Charles Oliver Nutter <charles.nutter@...> 2007/11/11

David Flanagan wrote:

[#13423] Re: method names in 1.9 — Jordi <mumismo@...> 2007/11/12

Summing it up:

[#13386] Re: method names in 1.9 — Trans <transfire@...> 2007/11/11

[#13391] Re: method names in 1.9 — Matthew Boeh <mboeh@...> 2007/11/11

On Sun, Nov 11, 2007 at 05:50:18PM +0900, Trans wrote:

[#13457] mingw rename — "Roger Pack" <rogerpack2005@...>

Currently for different windows' builds, the names for RUBY_PLATFORM

13 messages 2007/11/13

[#13485] Proposal: Array#walker — Wolfgang Nádasi-Donner <ed.odanow@...>

Good morning all together!

23 messages 2007/11/14
[#13486] Re: Proposal: Array#walker — Wolfgang Nádasi-Donner <ed.odanow@...> 2007/11/14

A nicer version may be...

[#13488] Re: Proposal: Array#walker — Trans <transfire@...> 2007/11/14

[#13495] Re: Proposal: Array#walker — Trans <transfire@...> 2007/11/14

[#13498] state of threads in 1.9 — Jordi <mumismo@...>

Are Threads mapped to threads on the underlying operating system in

30 messages 2007/11/14
[#13519] Re: state of threads in 1.9 — "Bill Kelly" <billk@...> 2007/11/14

[#13526] Re: state of threads in 1.9 — Eric Hodel <drbrain@...7.net> 2007/11/14

On Nov 14, 2007, at 11:18 , Bill Kelly wrote:

[#13528] test/unit and miniunit — Ryan Davis <ryand-ruby@...>

When is the 1.9 freeze?

17 messages 2007/11/14

[#13564] Thoughts about Array#compact!, Array#flatten!, Array#reject!, String#strip!, String#capitalize!, String#gsub!, etc. — Wolfgang Nádasi-Donner <ed.odanow@...>

Good evening all together!

53 messages 2007/11/15
[#13575] Re: Thoughts about Array#compact!, Array#flatten!, Array#reject!, String#strip!, String#capitalize!, String#gsub!, etc. — "Nikolai Weibull" <now@...> 2007/11/15

On Nov 15, 2007 8:14 PM, Wolfgang N=E1dasi-Donner <ed.odanow@wonado.de> wro=

[#13578] Re: Thoughts about Array#compact!, Array#flatten!, Array#reject!, String#strip!, String#capitalize!, String#gsub!, etc. — Michael Neumann <mneumann@...> 2007/11/16

Nikolai Weibull schrieb:

[#13598] wondering about #tap (was: Re: Thoughts about Array#compact!, Array#flatten!, Array#reject!, String#strip!, String#capitalize!, String#gsub!, etc.) — "David A. Black" <dblack@...> 2007/11/16

Hi --

[#13605] Re: wondering about #tap (was: Re: Thoughts about Array#compact!, Array#flatten!, Array#reject!, String#strip!, String#capitalize!, String#gsub!, etc.) — Trans <transfire@...> 2007/11/16

[#13612] Re: wondering about #tap (was: Re: Thoughts about Array#compact!, Array#flatten!, Array#reject!, String#strip!, String#capitalize!, String#gsub!, etc.) — "David A. Black" <dblack@...> 2007/11/16

Hi --

[#13624] Re: wondering about #tap (was: Re: Thoughts about Array#compact!, Array#flatten!, Array#reject!, String#strip!, String#capitalize!, String#gsub!, etc.) — "Nikolai Weibull" <now@...> 2007/11/16

On Nov 16, 2007 12:40 PM, David A. Black <dblack@rubypal.com> wrote:

[#13632] Re: wondering about #tap — David Flanagan <david@...> 2007/11/16

David A. Black wrote:

[#13634] Re: wondering about #tap — "David A. Black" <dblack@...> 2007/11/16

Hi --

[#13636] Re: wondering about #tap — "Rick DeNatale" <rick.denatale@...> 2007/11/16

On Nov 16, 2007 12:40 PM, David A. Black <dblack@rubypal.com> wrote:

[#13637] Re: wondering about #tap — murphy <murphy@...> 2007/11/16

Rick DeNatale wrote:

[#13640] Re: wondering about #tap — Wolfgang Nádasi-Donner <ed.odanow@...> 2007/11/16

murphy schrieb:

[#13614] Suggestion for native thread tests — "Eust痃uio Rangel" <eustaquiorangel@...>

Hi!

12 messages 2007/11/16

[#13685] Problems with \M-x in utf-8 encoded strings — Wolfgang Nádasi-Donner <ed.odanow@...>

Hi!

11 messages 2007/11/18

[#13741] retry semantics changed — Dave Thomas <dave@...>

In 1.8, I could write:

46 messages 2007/11/23
[#13742] Re: retry semantics changed — "Brian Mitchell" <binary42@...> 2007/11/23

On Nov 23, 2007 12:06 PM, Dave Thomas <dave@pragprog.com> wrote:

[#13743] Re: retry semantics changed — Dave Thomas <dave@...> 2007/11/23

[#13746] Re: retry semantics changed — Yukihiro Matsumoto <matz@...> 2007/11/23

Hi,

[#13747] Re: retry semantics changed — Dave Thomas <dave@...> 2007/11/23

[#13748] Re: retry semantics changed — Yukihiro Matsumoto <matz@...> 2007/11/23

Hi,

[#13749] Re: retry semantics changed — Dave Thomas <dave@...> 2007/11/23

hacking on the "heap" implementation in gc.c

From: Lloyd Hilaiel <lloyd@...>
Date: 2007-11-01 19:09:53 UTC
List: ruby-core #13161
Hi all,

I've been playing (and that's all it is at this point) with the heap  
implementation in gc.c.  In practice the behavior I see with ruby 1.8  
today is that the heaps grow, but do not shrink.  My theory was that  
large heaps are bad, because they make it impossible to ever reclaim  
heap space (in fork case, or programs that do a lot of processing,  
then calm down, i.e. PList parsing).

A large heap is bad because of the scanning we do.  All pages must be  
resident and private (copy on write).  I know some folks are looking  
at addressing the copy on write issues with external storage of mark  
bits.  I'm hoping that some more fundamental changes which decrease  
memory usage without affecting performance may offer benefits which  
are valuable with or without moving gc mark bits out of the heap.

So the changes I've made are:
1. break heap implementation out from gc.c into heap.c and heap.h
2. reduce heap size to 0x400 bytes
3. optimize common operations on heaps to repair performance.

Once the above is done, it's no longer meaningful to say "do GC once  
before we add a heap", because heaps are tiny.  So instead I've a  
simple heuristic in newobj that says, "perform next garbage  
collection when the number of slots used rises to 180%".    
(preserving the magic number of 1.8, at least in spirit.  Finally,  
I'm experimenting with having this magic number be adaptive based on  
the fragmentation of the heap (% free). so it's 1.2 + (1 - (%free/ 
100)).  if 50% of the heap is in use, we'll GC again at 170%, if 20%  
is free, we'll wait till the slots used doubles.  The idea here is  
that the more fragmented the heap the higher the potential that we  
can reclaim space (compact) by running GC more frequently.  On  
average, approximately the same number of GCs are done in a run as  
with upatched 1.8.6, they are just done at different times which  
seems to be a good thing so far...

Some preliminary results and my patches are available here:
http://lloydforge.org/projects/ruby

What I would love is criticism on the approach, and some performance  
testing of this patch to see if the gains I'm seeing are artificial.   
This patch is just a proof of concept and play, many things would  
need to be cleaned up before it would be something that could be  
useful (RUBY_CRITICAL being one thing of many).

hope this is interesting,
lloyd




In This Thread

Prev Next