From: james@...
Date: 2002-06-20T15:12:46+09:00
Subject: RE: REXML in C

>
> It seems like the claim that I should have to give up the nice
> interface just because the problem gets large is fundamentally flawed
> somehow.  The acceptance of such rules of thumb is surely one of the
> reasons why the XML world sucks so much.

The XML world may or may not suck, but this rule of thumb has nothing to do
with XML per se.  It's a general observation that loading a large object
into memory before performing any work on it may not be an optimum choice.
For example, if you wanted to do a simple search and replace on a 13MB text
file, would it be reasonable to load it into an in-memory structure and
then call gsub?  Or might it be better to stream the file and work on small
subsets as they pass?


>
> The files are only 13MB or so, which isn't even large by new PC
> standards.


If you consider that an acceptable size, then there's really no problem.
Keep in mind that the file sizes may change, and not all APIs are designed
to scale for all sizes.  An in-memory structure API requires a trade-off
between immediate access to all parts of an object versus the cost of
holding that object in memory.  The value of that tradeoff will vary as the
problem size varies.


James