From: "Ara.T.Howard" <ahoward@...>
Date: 2004-06-09T08:23:37+09:00
Subject: Re: ruby and mustard

On Wed, 9 Jun 2004, Jeff Mitchell wrote:

> Is it possible to somehow isolate the inner-loop functionality?  As long
> as you can stay away from raw iterations of every data point -- calling only
> row or column operations, for example -- you have a pretty good chance of
> being fast enough.

not really.  i was trying out a combo of mmap an narray:

   mmap = Mmap.new 'huge', 'rw', Mmap::MAP_SHARED
   na = NArray.to_na mmap.to_s, NArray::BYTE
   positions = (na.eq val).where

(how cool is it that this works!)

but this blows the top right off memory/swap.  eg.  if i could to this

   (na.eq val).where do |pos|
     ...
   end

eg.  iff #where took a block i could do it this way.  any sort of collection
will blow up since i'm looking for potentially 2 ** 30 positions and streaming
them on stdout (to another program) and each of those positions can occupy
(guessing) 30 bytes or so (how big is '1234567' in ruby?)...  in otherwords i
really do have to handle them one at the time.

i can hear you thinking... why doesn't he mmap a peice at a time and use some
sort of buffering?

see

   https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=124624

for the reason why...


> Don't forget you can call any function in any shared library with ruby/dl,
> including memcpy and so fourth.  Make some trivial extension classes which
> simply hold raw chunks of data in a C array, or just pack strings.  You
> can then pass this data to C lib functions or ruby extensions.

ah,  ruby/dl - i've never used this.  know of any good examples?  this __is__
interesting.

-a
--
===============================================================================
| EMAIL   :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE   :: 303.497.6469
| A flower falls, even though we love it; and a weed grows, even though we do
| not love it. --Dogen
===============================================================================