From: Charlie Bowman <charlie@...>
Date: 2006-03-02T02:21:27+09:00
Subject: Re: [Newbie] Getting data from html-ish like crap.

--=-7mQ6BB4fWm/p5PEspuGC
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit

Here's some sample code you might enjoy.  It's a random chuck norris
joke generator that pulls the jokes off of a website.

require 'net/http'


page='http://www.4q.cc/index.php?pid=fact&person=chuck'
res = Net::HTTP.get(URI.parse(page))
res.scan(/(<\/h1>)(.*)(<hr \/>)/)
puts ($2 || 'No fact was found!')


On Thu, 2006-03-02 at 01:43 +0900, Justin Bailey wrote:

> First, don't use Net::HTTP. Require 'open-uri' at the top and you can
> simplify your code a lot:
> 
> open(https://www.knightonlineworld
> >
> > .com/index.php?pg=rankings&sub=2&radServer=1&clanid=12199) do |page|
> 
> 
> html = page.gets(nil)
> end
> 
> 
> Which will get the whole document into the 'html' variable.
> 
> Next, look at StringScanner and using regular expressions. It will allow you
> to iterate through your document quickly and pick up the columns you want.
> Some pseudo-code might look like:
> 
> scanner = StringScanner.new
> while scanner.check(/<tr>.*<td>(.*)<\/td>.*<td>.*<\/td>.*<td>.*<\/td>.*<td>(.*)<\/td>.*</tr>/m)
> do
>   name = scanner[1]
>   points = scanner[2]
> end
> 
> That will extract the name and level from each row. The parantheses in the
> regular expression are "capture groups", and they relate to the assignments
> in the loop (name = scanner[1], points = scanner[2]). The 'm' following the
> regular expression makes sure a multi-line match is performed, which is
> probably necessary as the table cells are on different lines.
> 
> For further syntax and library help check http://www.ruby-doc.org.
> Especially check out the 'Programming Ruby' book and read up Ruby's regular
> expressions, if you aren't familiar with them.
> 
> Hope that helps!
> 
> 
> On 3/1/06, spam_monkey <"sidhellfire(spam_monkey)"@o2.pl> wrote:
> >
> > Hi,
> > I wanted to learn something, and choosed ruby,
> > since it looked awesome, and can't say it isn't.
> > I am not expierienced programmer (tried some
> > Pascal, then PHP), and decided to do something
> > small, but usefull. Let's get straight into the
> > problem.
> >
> > At the url:
> >
> > https://www.knightonlineworld.com/index.php?pg=rankings⊂=2&radServer=1&clanid=12199
> > I've got something similar to html (at least
> > it's not table based, but still damn ugly code
> > there) with online statistics.
> >
> > I don't want to parse that one, i just want to
> > 'crop that crap' and retrive informations from
> > data inside one element a div with an
> > id="bleet". There are tables in there, but seems
> > impossible to navigate there.
> >
> > I am really suck at strings :(
> >
> > <tr bgcolor="#FFFFFF">
> > <td align="center" >Quatrina</td> # i want this
> > <td align="center" >52</td>       #
> > <td align="center" >Shaman</td>   #
> > <td align="center" >563</td>      # and this one
> > </tr> # preferable everything :P
> >
> > I would want to hash that data, to have it
> > usefull in future (aiming a rails app in
> > future), but selecting two columns in each row,
> > located in the last table in the id'ed element
> > placed in not-well-made document looks
> > impossible for me. I don't even know where to
> > start, and it's FAR away from the things i
> > wanted to do (counting numbers*, assinging
> > additional data).
> >
> >
> > All i've done already is getting the document:
> >
> > > require 'uri'
> > > require 'net/http'
> > >
> > > trg = "
> > https://www.knightonlineworld.com/index.php?pg=rankings⊂=2&radServer=1&clanid=12199
> > "
> > > puts 'processing' + cel + " :\n"
> > >
> > > r = Net::HTTP.get_response(URI.parse(trg).host, URI.parse(trg).path)
> > >
> > > puts r.body
> >
> >
> > Thanks for reading.
> >
> > */Notice it does count "Loyality" wrong/
> >
> >

Charlie Bowman
http://www.recentrambles.com

--=-7mQ6BB4fWm/p5PEspuGC--