From: Wincent Colaiuta <win@...>
Date: 2008-02-07T02:29:56+09:00
Subject: Avoiding memcpy() during String creation in a C extension

I'm working on a fast wikitext-to-HTML translator written in C and am
trying to optimize it.

Profiling shows that a fair portion of the time is being spent
instantiating temporary strings and much of that time is spent in
memcpy.

In much of the code I could avoid many String instantiations and just
prepare everything in a buffer at the raw byte level, not touching any
Ruby objects, and do a rb_str_new() at the end.

The problem with this approach is that Ruby will alloc space for the
string and then memcpy the buffer, which is wasteful seeing as I've
already done the alloc and I just want to relinquish control of that
block of memory and hand it over to Ruby.

Basically what I would like is a String instantiation method which
could take a pointer to a buffer and basically take ownership of that
buffer instead of calling memcpy() on it.

I've looked in string.c to see if such a function exists and it
appears that it does not. I know I could write my own such function
but that is starting to meddle with the internals a little bit too
much for my liking, and I'd need a separate version for Ruby 1.8 and
another for 1.9. It would be very brittle and could break if changes
are made to Ruby in the future.

Any ideas? Or should I just resign myself to that redundant memcpy()?
We're not talking about a lot of memory, but it's something that can
run thousands of times per second and is currently processing wikitext
at about 6 megabytes per second and I'd like to make it even faster.

Cheers,
Wincent