From: Stuart Clarke <stuart.clarke1986@...>
Date: 2010-09-04T21:10:12+09:00
Subject: Re: Slow Find.find - real problem

Thanks for getting back to me. At present I read the CSV file get the 
first line of information the. Search the file system in the object 
above.

Are you saying I should hash the file system, loop through the hash and 
get the first entry then search the csv and rename?

Could you provide an example please? The basenames are all unique!

Thanks a lot

Brian Candler wrote:
> Stuart Clarke wrote:
>> My issue is this is very very slow and is taking about an hour to do 10
>> file renames. Is there any way I can speed this up?
> 
> The slow bit will be scanning the filesystem.
> 
> I suggest you run a single File.find across the whole tree, building a 
> suitable data structure (e.g. a hash of {basename => full filename} for 
> every file which is eligible to be renamed)
> 
> Then when you read in the CSV, you can just cross-reference to this data 
> structure and locate the files immediately.
> 
> If there is a possibility of more than one file with the same basename 
> existing in multiple directories, then build your data structure 
> appropriately, e.g.
> {basename => [fullpath1, fullpath2, fullpath3, ...]}
> 
> Alternatively, do this the other way round: read in the whole CSV (which 
> is likely to be fast) into a suitable data structure, and then scan 
> across the filesystem once; while you scan it, for every file you find 
> check whether the CSV had a rename instruction for it.
> 
> If the CSV is too big to fit into memory then consider loading it into a 
> database of some sort instead.
> 
> HTH,
> 
> Brian.

-- 
Posted via http://www.ruby-forum.com/.