From: Timothy Hunter <cyclists@...>
Date: 2005-09-01T09:16:25+09:00
Subject: Re: rmagick question

Ara.T.Howard wrote:
> On Wed, 31 Aug 2005, Timothy Hunter wrote:
> 
>> Darn. After reading Ara's post I started thinking about a way to move 
>> pixels
>> more-or-less directly from a file into an image, bypassing the need to
>> create Magick::Pixel objects. I emailed the ImageMagick team and asked 
>> for
>> their recommendation. The absolute fastest way to get pixel data into an
>> image is via the ImageMagick API that RMagick's Image::import_pixels 
>> method
>> (http://www.simplesystems.org/RMagick/doc/image2.html#import_pixels) 
>> uses,
>> called ImportImagePixels. ImportImagePixels expects a C array of pixel 
>> data
>> in scanline order, top-to-bottom, right-to-left. You can specify the data
>> type of the array (char, short, int, long) and the order in which the
>> channel data appear (RGB, RGBA, CMYK, etc.) The IM developers tell me 
>> that
>> ImportImagePixels is optimized for the RGB case so I suspect that this 
>> would
>> be a very - I repeat, very - fast way to load pixels into an image.
> 
> 
> plus plus on the 'very' ;-)
> 
>> Currently #import_pixels wants a Ruby array of Fixnums which it then
>> converts to a C array. Lotsa overhead.
>>
>> Following Ara's suggestion, I was thinking about changing 
>> #import_pixels to
>> accept a string (to be exact, any object that responds to to_str) in 
>> place
>> of the array. In this case #import_pixels would simply call to_str and
>> assume that the result is a C array of the correct type and size and with
>> the channel data in the specified order and hand it off to 
>> ImportImagePixels
>> directly.
> 
> 
> so
> 
>   pixels, is_string =
>     case obj
>       when Array
>         [ obj, false ]
>       else
>         [ obj.to_str, true ]
>     end
> 
>   ...
>   ...
> 
> ??
> 
>> I realize this doesn't really help you since your input data isn't in the
>> format ImportImagePixels wants. However, if you (or Ara, or anybody else
>> with an interest) have an opinion about the usefulness of this idea 
>> I'd like
>> to hear it.
> 
> 
> it sounds brilliant!  but:
> 
>     jib:~ > irb -r mmap -r narray
> 
>     irb(main):001:0> NArray::byte(42,42).methods.grep /to_s/
>     => ["to_s", "to_string"]
> 
>     irb(main):002:0> Mmap::new('/home/ahoward/.bashrc').methods.grep /to_s/
>     => ["to_sym", "to_str", "to_s"]
> 
> so maybe something like
> 
>   cast = %w( pixels to_str to_s to_string ).select{|m| obj.respond_to? m 
> }.first
> 
>   pixels = obj.send(cast) if cast
> 
> eg.  include other likely candidates in addition to to_str.  i think 
> it's safe
> to say that any user capable of setting up a region of memory 
> representing an
> inline image will be willing to accept any consequences of doing it 
> improperly
> so doing a blind read is fine in this case - but that's obviously my 
> opinion.

Actually I was thinking that NArray would meet me half-way.

Right now #import_pixels calls Kernal.Array on the `pixels' argument. My 
reasoning for doing this is that this approach allows the caller to pass 
any object that supports #to_ary or #to_a (including NArray objects) to 
#import_pixels. (The downside for NArrays, of course, is that NArray 
responds to #to_a by constructing a real Ruby array with a zillion 
elements.)

Now I want to filter out strings and treat them specially. Currently a 
string is not a reasonable argument since Kernel.Array simply constructs 
an array with the string as its only element. Not too useful for 
building images. My notion was to do something like this:

if pixels.respond_to?(:to_str)
	pixel_buffer = pixels.to_str
	# pass the buffer directly to ImageMagick
else
	pixel_array = Kernel.Array(pixels)
	# convert the array to a buffer and pass it to IM
end

I can't think of a way this would break existing code, can you? You 
could use the return value from NArray#to_s, mmap#to_str, or IO.read as 
the `pixels' argument.

The upside is that for real String objects, #to_str is a no-op. The 
downside, at least for NArray objects, is that #to_s makes a copy of the 
  data in the NArray. I've perused the NArray source code and I didn't 
find a way to directly access the NArray data without making a copy.

One more complication. The ImportImagePixels function in ImageMagick 
requires an argument that identifies the type of type of the data in the 
pixel buffer as char (8-bit), short (16-bit) or int (32-bit). 
ImageMagick will convert the data as necessary to the size it needs. It 
seems to me that it would be useful to support the use of data that is 
not the same size as the underlying pixel type. That is, you could 
reasonably want to construct an image with 8-bit pixel data from an 
NArray.sint (16-bit) object, or vice versa. So, I'm thinking that 
#import_pixels should accept an optional 7th argument that indicates the 
type of the incoming data (CharPixel, ShortPixel, LongPixel enum values, 
probably). The default would be CharPixel. This would also make it 
possible to use the same script with different configurations of 
ImageMagick.

Thoughts? I'll hold off writing any code until we're in agreement.


> 
> thanks for keeping tabs on this btw.
> 

Thanks for the idea!