From: Chilkat Software <support@...>
Date: 2006-11-08T13:09:45+09:00
Subject: Re: REXML


Thanks,

By the way, the Chilkat XML parser is not better than
REXML, it's just different.  To give you a little history, it was originally
developed about 7 years ago to handle:

1) Large XML data files where the MSXML parser was s-l-o-w.
At one point, I remember Chilkat XML parsing files in a few seconds
that took MSXML minutes to parse.  However, since then MSXML has
improved to the point where it's as good or better in speed...

2) I wanted to create a parser that was forgiving with errors.
Back then, it was a nightmare to have a large XML file with one
small error, perhaps a byte or two that didn't fit the charset encoding,
that would prevent the entire document from loading.

3) I wanted a parser that made it easy to do the common tasks
I'm always faced with in XML -- such as reading/writing config files.

4) I wanted to make it easy to do things not normally handled in
an API -- sorting, compression, encryption, loading / encoding binary 
data, etc.

If you give it a try -- let me know what you think.  Send me a request for
an example or two and I'll be happy to provide what I can...

-Matt

At 09:30 PM 11/7/2006, you wrote:

>Figured it out.
>
>It was running into problems with videos and podcasts as they don't
>have an artist tag.
>Got round it by searching for children tags with the title of "Movie"
>or "Podcast", and not running the print statements for those items.
>
>
>pdg wrote:
> > Hi thanks for the link, it seems to be working much better, but...
> >
> > It's getting to about the 1000th file and doing its job, but then
> > returning the following error:
> >
> > undefined method 'NextSibling' for nil:NilClass (noMethodError) from
> > rtunes rb (which is basically just your sample code).
> >
> > Does this mean I have a broken xml file? Or is something else the
> > matter?
> >
> >
> > Chilkat Software wrote:
> > > I tested the Chilkat XML parser (an in-memory DOM) on a 21MB XML file
> > > that looks like this:
> > >
> > > <phonebook>
> > > 
> <address><company>yuy25uiFfaku</company><street>A7ZbA3jP48rp</street><city>fSgWAhn3i3lD</city><state>p3rfNqf6kzUq</state><postal_code>lqVZ0b4daYWQ</postal_code><country>VjfXvb0AdxSt</country><extra>TEST</extra></address>
> > > 
> <address><company>Ki78Ypx8FlbZ</company><street>340PK6u2DsZQ</street><city>EqbFawBo0mCi</city><state>fTZK5YT0Tur8</state><postal_code>EXP29c5Hi2Hj</postal_code><country>sfGB4EzWR3Ft</country><extra>TEST</extra></address>
> > > ..
> > > </phonebook>
> > >
> > > (the data is random garbage...)
> > >
> > > The XML is parsed in 11.5 seconds on a 18.Ghz Pentium 4.  Peak memory
> > > usage is 180MB.
> > > I don't think the parser would break a sweat on the 6MB file...  ;-)
> > >
> > > I uploaded the XML test data to:
> > > http://www.example-code.com/downloads/bigXml.zip
> > > The code for parsing the iTunes XML is easy:
> > > http://www.example-code.com/ruby/ruby-parse-itunes-xml.asp
> > >
> > > -Matt
> > >
> > >
> > >
> > >
> > > At 02:45 PM 11/7/2006, you wrote:
> > >
> > > >Hi David (or others)
> > > >
> > > >I am still not sure I get it, could you explain a little more?
> > > >
> > > >Thanks,
> > > >Paul.
> > > >
> > > >David Vallner wrote:
> > > > > pdg wrote:
> > > > > > Assuming I go with the Ruby pull parser, how do I use 
> this in my code.
> > > > > > I see from the link the code sample, but I have no idea 
> how to throw
> > > > > > that into my code and make it work. Any suggestions.
> > > > > >
> > > > >
> > > > > Generally, you should have some layer between XML input, 
> and processing
> > > > > the records themselves. E.g. a trivial Song class, or at 
> least a hash.
> > > > > Personally, I'd make a XMLSongList class that's enumerable 
> (implements
> > > > > #each), and rework the REXML code that works for small files into one
> > > > > that yields a Song object for each of the records in succession by
> > > > > querying the tree accordingly.
> > > > >
> > > > > That shouldn't then be too hard to rework so that while #each is
> > > > > running, it opens a pull parser, and for each yield, builds up a Song
> > > > > object going through the record in the order how the 
> elements appear in
> > > > > the XML file, instead of a random one. Once you isolate the code that
> > > > > manipulates the XML to the smallest significant unit (a 
> song record in
> > > > > this case, I presume), it shouldn't be conceptually that difficult to
> > > > > rework from a tree parser to a pull parser. The code 
> probably will get a
> > > > > little messier and verbose, but the main shift of thinking is in not
> > > > > asking the XML for what your object needs, but feeding an object what
> > > > > the XML has.
> > > > >
> > > > > > PS: idiot (slaps head). Yes it was 5-6meg not gig!
> > > > > >
> > > > >
> > > > > 6MB is still Huge (tm) for a XML file.
> > > > >
> > > > >
> > > > > --------------enigF9D6B5236ACE2603700BC85A
> > > > > Content-Type: application/pgp-signature
> > > > > Content-Disposition: inline;
> > > > >       filename="signature.asc"
> > > > > Content-Description: OpenPGP digital signature
> > > > > X-Google-AttachSize: 188
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >--
> > > >No virus found in this incoming message.
> > > >Checked by AVG Free Edition.
> > > >Version: 7.1.409 / Virus Database: 268.13.31/522 - Release 
> Date: 11/7/2006
> > >
> > >
> > > --
> > > No virus found in this outgoing message.
> > > Checked by AVG Free Edition.
> > > Version: 7.1.409 / Virus Database: 268.13.31/522 - Release 
> Date: 11/7/2006
>
>
>
>
>
>--
>No virus found in this incoming message.
>Checked by AVG Free Edition.
>Version: 7.1.409 / Virus Database: 268.13.31/522 - Release Date: 11/7/2006


-- 
No virus found in this outgoing message.
Checked by AVG Free Edition.
Version: 7.1.409 / Virus Database: 268.13.31/522 - Release Date: 11/7/2006