From: Robert Klemme <shortcutter@...>
Date: 2006-11-04T18:00:07+09:00
Subject: Re: REXML ... performance & memory usage ...

David Vallner wrote:
> Jeff Wood wrote:
>> Wow ... I am trying to use REXML to parse through an 8.8Mb xml file ...
>> it currently eats almost 800Mb of ram before it seems to do anything ...
> 
> At that file size, I'd also slightly start thinking of biting into the
> bitter pill and using stream / pull parsing instead of tree parsing.
> Even with using a C parser a DOM buildup is not going to do much good
> for performance if you need to to processing at that scale more than
> seldom. But then again, there's the premature optimization quote that
> says to wait with that just yet.

I would not necessarily call that premature optimization.  If these 
kinds of files are to be parsed frequently and if only a portion of them 
needs extracting then I would also go down the stream parser road.

 From my experience stream parsers are also appropriate if you have to 
transform the XML tree of a document into some other object structure. 
IMHO the coding effort for transforming a DOM into another object tree 
vs. doing the same with the stream approach is quite equivalent.  And 
runtime wise you save yourself one whole tree traversal by going stream.

Kind regards

	robert