From: "ara.t.howard" <ara.t.howard@...>
Date: 2008-04-10T06:27:46+09:00
Subject: Re: heavy loop functions slow



> The dump would be to the pre-defined hash, to hence retrieve the
> information faster.
>
> To answer your 2nd question yes this is just a single step in a very
> large 12 step application. I'm hoping to condense it down to about 8
> steps when I finish.  This step alone involves transforming a large
> dataset into a smaller dataset.
>
> I'm trying to extract all the numbers between ranges and push the keys
> of the hash results into a file. This file will then be opened by
> another part of the step process to be analyzed.
>
> IE:
> if the transformation involved the file of:
> 12345
> 67423
> 97567
> 45345
> ect.
> I would want to pull all of those numbers and get the keys for those
> hash ranges
> IE:
> 12000..15000 => 100
> 60000..70000 => 250
> ect.
>
> So 12345 would fall in the range of 12000.15000 so the output file  
> would
> get 100 added to it. Then the next step would be analyzing the results
> (IE: 100).
> Hope this explains things a bit better.
>
> Thanks,
>



cfp:~ > cat a.rb
#
# use narray for fast ruby numbers
#
   require 'rubygems'
   require 'narray'

#
# ton-o-date
#
   huge = NArray.int(2 ** 25).indgen * 100 # 0, 100, 200, 300, etc

#
# bin data
#
#         0...100 -> 0
#       100...200 -> 1
#       200...300 -> 2
#       etc...
#

   a = Time.now.to_f

   p huge

   huge.div! 100  # 42 -> 0, 127 -> 1, 2227 -> 222

   b = Time.now.to_f

   elapsed = b - a

   p elapsed

   p huge



cfp:~ > ruby a.rb
NArray.int(33554432):
[ 0, 100, 200, 300, 400, 500, 600, 700, 800, 900, 1000, 1100, 1200,  
1300, ... ]
0.202844142913818
NArray.int(33554432):
[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18,  
19, ... ]



so that's doing about 33 million elements in around 2/10ths of a  
second....




a @ http://codeforpeople.com/
--
we can deny everything, except that we have the possibility of being  
better. simply reflect on that.
h.h. the 14th dalai lama