From: Michael Linfield <globyy3000@...>
Date: 2008-04-08T18:32:19+09:00
Subject: Re: heavy loop functions slow

Robert Klemme wrote:
> 2008/4/8, Michael Linfield <globyy3000@hotmail.com>:
>>  output2 = output[356131..712260]
>>     end
>>  Any ideas that would speed this up are much appreciated!! Otherwise I'll
>>  be back in 3 months IF I dont get an error :D
> 
> Obviously there is a lot of code missing from the piece above.  Can
> you explain, what you are trying to achieve?  What is your input file
> format and what kind of transformation do you want to do on it?  I
> looked through your other postings but it did not become clear to me.
> 
> Cheers
> 
> robert

Alright heres the breakdown of everything.


dataArray = []

# arrayOut consist of all integer data stored in a text file.
# its called upon via IO.foreach("data.txt"){|x| dataArray << x}
# dataArray being just a predefined array ie: dataArray = []

output = arrayOut.to_s.chop!.split(",")

#Each of these outputs breaks down this huge array into 4 smaller arrays
output1 = output[0..356130]
output2 = output[356131..712260]
output3 = output[712261..1068390]
output4 = output[1068391..1424521]

#hashRange[out] is basically calling a hash in the following context.
# hash = { 1=> { 20000..30000 => 12345 } }
#so 'out' is calling the range of the key to which contains its defined 
value
#basically its saying hashRange[25000]    #=> 12345   as an example

#everything imported to dataArray is a string, so it must be converted 
to an
#integer to correctly match the range key

#after benchmarking some elements of the loop below its found to be
#the push = hashRange[out] is whats slowing everything down.
#everything a nil 'out' is shoved into the query it takes about 8sec.
#when its a correct number, takes about 5sec

#the hashRange file is about 78mb, to which I had to load in as
#8 separate data files, then shove those into an eval to convert it
#to a hash

count = 0
    output1.each do |out|
      out = out.to_i
      push = hashRange[out]
      dataArray << push
      count+=1
      puts "#{push} - #{count}"     #Testing purposes
    end

#I guess what I need now is a faster way to access this pre-defined 
hash.
#SQL is one possibility but that could be considered a whole other
#forum post :)

Any other questions feel free to ask,
Your guy's insight is much appreciated.

Thanks again,

- Mac
-- 
Posted via http://www.ruby-forum.com/.