From: Ralph Shnelvar <ralphs@...32.com>
Date: 2010-08-23T00:37:27+09:00
Subject: Re: YAML::dump slow

------------781862441F1BDEB6
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit

Saturday, August 21, 2010, 4:56:43 PM, you wrote:

MF> On Sun, Aug 22, 2010 at 1:45 AM, Ralph Shnelvar <ralphs@dos32.com> wrote:
>> This is interesting and counterintuitive ...

>> I am creating a YAML file with about 200,000 entries.   The YAML file "reflects" the data structure of a directory tree of a disk.   All I have is directory names and subdirectories and filenames in the (sub) directories.  I have 100,000 simulated directories and 100,000 simulated files.

>> It takes about
>>   10 seconds to create this data structure in memory
>>   10 _minutes_ to create the YAML string (i.e. YAML::dump)
>>    1 second to write out the YAML string (25 megabytes)
>>    4 seconds to read the YAML string (why so long?)
>>    8 seconds to do the YAML::load

>> Anyone have any ideas how to speed up the creation of the YAML file or why there is such an asymmetric amount of time for YAML::dump and YAML::load?

MF> If you're on 1.9.2, you could try the new YAML library psych:
MF> iota ~ % time ruby -rpsych -e 'Psych.load(Psych.dump(Dir["/sys/**/*"]))'
MF> 1.20s user 0.07s system 99% cpu 1.283 total

MF> Of course, that's just a simple Array, but would be interesting to see
MF> how your data behaves.

Michael,

I am running
  ruby 1.8.7 (2010-01-10 patchlevel 249)


"YAML library psych" ????   What does that mean?
------------781862441F1BDEB6--