From: Reimer Behrends <behrends@...>
Date: 2002-08-21T01:30:30+09:00
Subject: Re: Data_Make_Struct Considered Dangerous?

William Djaja Tjokroaminata (billtj@y.glue.umd.edu) wrote:
>  Reimer Behrends <behrends@cse.msu.edu> wrote:
>  --------------------------------------------------
> > This is inaccurate. Ruby only knows about the memory allocated via
> > ALLOC(), not of memory allocated via malloc(). So, Ruby's heuristic
> > as to when to start the next GC might be way off, and you might
> > use more physical memory than necessary. In the extreme case, Ruby
> > would not start garbage collecting before you run out of swap space.
>  --------------------------------------------------
>  
>  Please see my response to Joel.  Isn't it that the maximum possible wasted
>  memory is determined by GC_MALLOC_LIMIT, which has default values of 8
>  megabytes?  As an example, suppose I have a total of 128 Mbytes of
>  memory.  My Ruby memory is 16.1 Mbytes and my C memory is 64
>  Mbytes.  Isn't it that the total used memory will be 24 + 64 = 88 Mbytes?

A problem occurs if, say, you have an Image class that allocates memory
for the pixel data using malloc():

for i = 1..1000 do
  img = Image.load(directory + i.to_s + ".png")
end

Assuming that each image occupies, say, 1 MB in memory, the program
allocates roughly 1 GB of memory. Yet as far as Ruby knows, only a few
kilobytes have been requested, so no garbage collection is initiated
and the program starts chewing through all the available swap space,
even though there is plenty of garbage to collect.

That may not be a problem with your specific application, but your
overall argument that ALLOC() can generally be replaced by malloc() does
not work, because memory may not be freed in a timely fashion. It sort
of works if the malloced parts are small relative to object size,
essentially increasing the amount of garbage that can exist in the heap
by a proportional factor.

In a nutshell, use of malloc() instead of ALLOC() can break the garbage
collection heuristic.

[...]
>  --------------------------------------------------
> > A big flaw in your argument is that you assume that you can always take
> > over the duties of freeing memory yourself. That works if the allocated
> > memory is referenced only by a single object. If the reference gets
> > passed around, you'll essentially find that you're implementing memory
> > management yourself again, trying to find out if the malloc()ed memory
> > is still accessible.
>  --------------------------------------------------
>  
>  Oh yes, because I am writing in C, memory management is part of everyday
>  life;  it is not like that in Ruby or Java.  About the references to Ruby
>  objects, aren't they taken care of by the mark functions?

I am not talking about references _to_ Ruby objects. I am talking about
several Ruby objects sharing the same structure (or part of one) on the
C side. Consider objects with copy-on-write semantics, for instance:
cloning such an object will not create a duplicate of the internal
structure, but the first attempt to modify it will.

>  --------------------------------------------------
> > Note that Data_Wrap_Struct() can initiate garbage collection as well,
> > namely in rb_newobj(). It doesn't really make a difference.
>  --------------------------------------------------
>  
>  Agreed.  The only difference is, we "reduce" the frequency by which the
>  garbage collector is invoked.  With Data_Make_Struct, the allocated memory
>  is the sum of the requested in ALLOC plus that in rb_newobj(); with
>  Data_Wrap_Struct, it is only rb_newobj() alone, which I hope is not too
>  large.

Your original argument was that Data_Make_Struct() is inherently unsafe;
but since Data_Wrap_Struct() can trigger a garbage collection as well,
there should be no difference in kind, just in degree. Efficiency is
another matter (note also that you may be paying the price of wasted
memory for fewer collection cycles).

			Reimer Behrends