From: pdg <pgattphoto@...>
Date: 2006-11-08T12:30:05+09:00
Subject: Re: REXML

Figured it out.

It was running into problems with videos and podcasts as they don't
have an artist tag.
Got round it by searching for children tags with the title of "Movie"
or "Podcast", and not running the print statements for those items.


pdg wrote:
> Hi thanks for the link, it seems to be working much better, but...
>
> It's getting to about the 1000th file and doing its job, but then
> returning the following error:
>
> undefined method 'NextSibling' for nil:NilClass (noMethodError) from
> rtunes rb (which is basically just your sample code).
>
> Does this mean I have a broken xml file? Or is something else the
> matter?
>
>
> Chilkat Software wrote:
> > I tested the Chilkat XML parser (an in-memory DOM) on a 21MB XML file
> > that looks like this:
> >
> > <phonebook>
> > <address><company>yuy25uiFfaku</company><street>A7ZbA3jP48rp</street><city>fSgWAhn3i3lD</city><state>p3rfNqf6kzUq</state><postal_code>lqVZ0b4daYWQ</postal_code><country>VjfXvb0AdxSt</country><extra>TEST</extra></address>
> > <address><company>Ki78Ypx8FlbZ</company><street>340PK6u2DsZQ</street><city>EqbFawBo0mCi</city><state>fTZK5YT0Tur8</state><postal_code>EXP29c5Hi2Hj</postal_code><country>sfGB4EzWR3Ft</country><extra>TEST</extra></address>
> > ..
> > </phonebook>
> >
> > (the data is random garbage...)
> >
> > The XML is parsed in 11.5 seconds on a 18.Ghz Pentium 4.  Peak memory
> > usage is 180MB.
> > I don't think the parser would break a sweat on the 6MB file...  ;-)
> >
> > I uploaded the XML test data to:
> > http://www.example-code.com/downloads/bigXml.zip
> > The code for parsing the iTunes XML is easy:
> > http://www.example-code.com/ruby/ruby-parse-itunes-xml.asp
> >
> > -Matt
> >
> >
> >
> >
> > At 02:45 PM 11/7/2006, you wrote:
> >
> > >Hi David (or others)
> > >
> > >I am still not sure I get it, could you explain a little more?
> > >
> > >Thanks,
> > >Paul.
> > >
> > >David Vallner wrote:
> > > > pdg wrote:
> > > > > Assuming I go with the Ruby pull parser, how do I use this in my code.
> > > > > I see from the link the code sample, but I have no idea how to throw
> > > > > that into my code and make it work. Any suggestions.
> > > > >
> > > >
> > > > Generally, you should have some layer between XML input, and processing
> > > > the records themselves. E.g. a trivial Song class, or at least a hash.
> > > > Personally, I'd make a XMLSongList class that's enumerable (implements
> > > > #each), and rework the REXML code that works for small files into one
> > > > that yields a Song object for each of the records in succession by
> > > > querying the tree accordingly.
> > > >
> > > > That shouldn't then be too hard to rework so that while #each is
> > > > running, it opens a pull parser, and for each yield, builds up a Song
> > > > object going through the record in the order how the elements appear in
> > > > the XML file, instead of a random one. Once you isolate the code that
> > > > manipulates the XML to the smallest significant unit (a song record in
> > > > this case, I presume), it shouldn't be conceptually that difficult to
> > > > rework from a tree parser to a pull parser. The code probably will get a
> > > > little messier and verbose, but the main shift of thinking is in not
> > > > asking the XML for what your object needs, but feeding an object what
> > > > the XML has.
> > > >
> > > > > PS: idiot (slaps head). Yes it was 5-6meg not gig!
> > > > >
> > > >
> > > > 6MB is still Huge (tm) for a XML file.
> > > >
> > > >
> > > > --------------enigF9D6B5236ACE2603700BC85A
> > > > Content-Type: application/pgp-signature
> > > > Content-Disposition: inline;
> > > >       filename="signature.asc"
> > > > Content-Description: OpenPGP digital signature
> > > > X-Google-AttachSize: 188
> > >
> > >
> > >
> > >
> > >
> > >--
> > >No virus found in this incoming message.
> > >Checked by AVG Free Edition.
> > >Version: 7.1.409 / Virus Database: 268.13.31/522 - Release Date: 11/7/2006
> >
> >
> > --
> > No virus found in this outgoing message.
> > Checked by AVG Free Edition.
> > Version: 7.1.409 / Virus Database: 268.13.31/522 - Release Date: 11/7/2006