From: Roger Pack Date: 2007-11-11T06:29:21+09:00 Subject: Re: enterprise ruby >> As I understand it, the problem is that MRI keeps some unused memory >> allocated and then the GC marks it dirty... So technically there's >> information being used and re-used frequently but only by the GC :-( > > > Well ... that sounds like an actual bug rather than a design issue in > MRI. Is it that the GC can't tell it's unused? The GC's mark and sweep 'recreates' its freelist every time it runs a GC, so if you have a lot of free objects (believe it or not), it will remark them all--possibly in about the same order as the previous. A design thing. So this interesting point of yours may have two implications: a ruby proc that retains lots of 'free' memory will have a longer sweep time (which having lotsa free is quite common with the standard MRI--it allocates exponentially larger and larger heap sizes, so you're almost guaranteed (with a large process) the the 'last most' heap will be half used, and, as you noted, the entire thing constantly remarked for every GC (all used marked as 'valid', all free remarked for the freelist). The way to avoid this would be to 'only add' to the freelist as you unallocate objects. Then you'd avoid marking the free objects. You could still free heaps the same way. If you did that you'd still be traversing them for every GC (to look through for allocated objects no longer accessible--unmarked objects), but wouldn't be marking them dirty. Drawback might be a freelist that isn't 'optimized in order' or something (probably not much of a drawback). Another way to kind of combat this is to use a smaller 'heap chunk' size (instead of Ruby's exponentially growing one), as this allows chunks more frequently to be freed, which means they aren't traversed (basically you don't have as much free memory kicking around, so you don't traverse it as much). It still leaves all free memory to traverse, however. If you wanted to avoid ever accessing freed objects at all, you'd need to create an 'allocated' list, as well, so you could just traverse the allocated list and then add those to the freelist that were freed. So about 20%/object size increase. Maybe a good trade off??? Tough to tell. If I guessed I'd say that the trade off is...worth it for large long standing processes. It would use more RAM and be faster. Maybe an optimized GC might not be such a bad idea after all :) -Roger -- Posted via http://www.ruby-forum.com/.