From: Eric Wong <normalperson@...>
Date: 2014-08-29T07:49:45+00:00
Subject: [ruby-core:64633] Re: cowspace (work-in-progress)

SASADA Koichi <ko1@atdot.net> wrote:
> Some comments:
> 
> Basically, great points.
> 
> (2014/08/28 18:10), Eric Wong wrote:
> 
> > The idea is to have a separate malloc
> space for long-lived, WORM (write once, read many) data which may
> increase memory sharing for forked processes.
> 
> Basically, "Write once" is important. Not for read many, isn't it?

Right.  WOLL (Write Once, Long Lived) is better term

> Maybe there are several abilites for memory blocks:
> 
> - life time        short - long  - eternal
> - write frequency  many  - a few - once
> - read many        many  - a few - once - nothing
> - size             variable size - fixed size
> 
> String buffer seems [lifetime: short, write: a few, read: many, size:
> variable]. So String is out of scope.

Maybe loaded pathnames and larger frozen strings can benefit
(frozen messages/warnings/help text)

> Current rb_iseq_t is [lifetime: long, write: a few, read: a few (depends
> on which method), size: fixed].
> 
> Bytecode refered from rb_iseq_t is [lifetime: long, write: once, read:
> a few (depends...), size: variable].
> 
> fixed size block can be aggregate (with another space with mspace or
> buffering technique) to avoid fragmentation.

Right.  I think one problem with dlmalloc is the consolidate/split
code ends up writing too many areas and triggering CoW.
I need to investigate.

Fixed size allocation can avoid that problem, too, but we have many
variable length allocations...

> So I assume the target of this API is such as bytecode, isn't it?

Yes, I but I also tried changing iseq->iseq and iseq->iseq_encoded
(forgot to publish) but didn't notice improvements...

> > The mspace API is similar to normal malloc:
> > 
> > 	malloc(size) => mspace_malloc(GET_VM()->cowspace, size)
> > 	free(ptr) => mspace_free(GET_VM()->cowspace, ptr)
> > 	...
> 
> Does mspace API supports to detect CoW breaking *writing*?
> Or programmer should predicate that this area is WORM area?

Programmer must declare area as WORM area; mspace API has no special
support for CoW, it is just a different API for private malloc pool.

> Another consideration is that it seems depends on dlmalloc. How is
> portability? (sorry I don't know about dlmalloc library more).

Looking at the dlmalloc code, it seems to support Windows/OSX.
I think it has a good reputation for portability.

> > However, I haven't found any benefits, yet.  _Maintaining_
> > CoW-friendliness is difficult in long-term so it might not be worth it
> > (compared to overall memory reductions).
> 
> For ISeq, we need to change ISeq data structure to avoid write. I
> believe this is my task.

OK.  I was hoping to avoid the size doubling with
iseq->iseq + iseq->iseq_encoded, too (I haven't investigated much)