From: Joe Van Dyk <joevandyk@...>
Date: 2005-08-31T07:36:21+09:00
Subject: Re: rmagick question

On 8/30/05, Timothy Hunter <cyclists@nc.rr.com> wrote:
> Joe Van Dyk wrote:
> > On 8/29/05, Joe Van Dyk <joevandyk@gmail.com> wrote:
> >
> >>On 8/29/05, Timothy Hunter <cyclists@nc.rr.com> wrote:
> >>
> >>>Joe Van Dyk wrote:
> >>>
> >>>>On 8/28/05, Timothy Hunter <cyclists@nc.rr.com> wrote:
> >>>>
> >>>>
> >>>>>No matter which approach you take let me know how it goes so I'll be
> >>>>>able to make recommendations to other RMagick users who are working with
> >>>>>very large images. Thanks!
> >>>>
> >>>>
> >>>>Yes, my Ruby program was taking up about 500 MB of memory (for a
> >>>>8700x6000 pixel image).  Memory's not a problem though, all of our
> >>>>machines have more than 2 gigabytes.
> >>>>
> >>>
> >>>Okay, first recommendation: have a honkin' great big machine :-)
> >>
> >>(in case people forgot, the color values in @tad_data are from 0-255,
> >>so conversion is needed if ImageMagick is using 16 bit color pixels)
> >>
> >>The inner code loop looked something like this:
> >>
> >>    (height - 1).downto(0) do |n|
> >>      # Code here that displayed percent-done status to user
> >>      width.times do |e|
> >>        # Lookup approach with 16 bit pixel ImageMagick
> >>        #rgb = @tad_data.read(8).unpack(TAD_FORMAT).collect! { |c|
> >>lookup_table[c] }
> >>
> >>        # Bitshifting approach with 16 bit pixel ImageMagick
> >>        #rgb = @tad_data.read(8).unpack(TAD_FORMAT).collect! { |c| (c
> >><< 8) | c }
> >>
> >>        # 8 bit ImageMagick, no conversion necessary
> >>        rgb = @tad_data.read(8).unpack(TAD_FORMAT)
> >>
> >>        pixel = Magick::Pixel.new(*rgb)
> >>        @image.pixel_color(e, n, pixel)
> >>      end
> >>    end
> >>
> >>Doing a 10000x10 pixel image (only 10 pixels high for unit testing
> >>purposes), I could process:
> >>
> >>13k pixels/second with the bitshifting approach (and 16 bit IM)
> >>13k pixels/second with the array lookup approach (and 16 bit IM)
> >>21k pixels/second with 256 bit IM.
> >>
> >>I also found that building up an array of colors for one row and then
> >>doing a @image.import_pixels had no speed improvements and the code
> >>was uglier.
> >
> >
> > Hm... I don't think that those pixels/second numbers are correct.  I
> > was doing the timings inside a Benchmark.measure { ... } block and
> > perhaps that makes things slower?
> >
> > I was able to process a 8700x6000 pixel image when using 'time ./script.rb'
> > 440.005u 9.917s 8:27.20 88.7%       0+0k 0+0io 52012pf+0w
> >
> > So, around 7 minutes for 5 million pixels.. that's around 118000
> > pixels per second.  Hm.  Oh well, it's fast enough for me.  This
> > operation isn't done all that often.   Although, a while ago I did
> > something like this in C with the GD library, and it took around 10-20
> > seconds to process a similar sized image.
> >
> >
> 
> Joe, what is the format of the pixel data in your input file? Is it just
> 8-bit R,G,B values in top-down scanline order?

Nope.  The data starts in the SW corner, then goes east and then
north.  The data for each pixel is packed into 8 bytes (a short for
elevation, a char for terrain info, a char for monochrome brightness,
and a network long that contains the color data).  Don't ask me why.