On Fri, Apr 5, 2013 at 7:57 AM, Soichi Ishida <= ;lists@ruby-forum= .com> wrote:

Rails 1.9.3

I am scraping a web page,
http://en.wikipedia.org/wiki/List_of_birds_by_common_name<= br>
using Nokogiri and Xpath

I want all the names of the birds listed there.

xpath1 =3D '//div[@id=3D'bodyContent']/div[@id=3D'mw-conten= t-text']/ul[1]'

That does= not work. =A0You need to escape single quotes or use double quotes.

<= div style> =A0

should get a group of birds

Great Tinamou
Andean Tinamou
Elegant Crested Tinamou
Little Tinamou
Slaty-breasted Tinamou
Thicket Tinamou

I need to get each one of these names and tried nested loop

xpath1 =3D '//div[@id=3D'bodyContent']/div[@id=3D'mw-conten= t-text']/ul'
doc =3D Nokogiri::HTML(open(url))
categories =3D doc.xpath(xpath1)

categories.each do | c |
=A0 =A0 c.xpath('/li').each do | n |
=A0 =A0 =A0 =A0 =A0p n.text
=A0 =A0 end
end

gives empty values.
Can anyone tell why?

Yes, your XPath = in the nested loop searches a <li> at _the top of the document_ becau= se you prefix with "/". =A0You would need

=A0

=A0or are there better ways?

Yes. =A0Having a loop here does not really make sense since that can be so= lved by the XPath.

irb(main):009:0> puts dom.xp= ath('//div[@id=3D"bodyContent"]/div[@id=3D"mw-content-te= xt"]/ul[1]//a/text()').map(&:to_s)

Great Tinamou

Andean Tinamou

Elegant Crested Tinam= ou

Little Tinamou

Slaty-breasted Tinamou

Thic= ket Tinamou

=A0

Note, why #to_s is necessary to get St= ring instances:

irb(main):012:0> dom.xpath('//div[@id=3D"bodyC= ontent"]/div[@id=3D"mw-content-text"]/ul[1]//a/text()').= each {|n| p n}

#<Nokogiri::XML::Text:0x..fc02b6340 "Gre= at Tinamou">

#<Nokogiri::XML::Te= xt:0x..fc02b469e "Andean Tinamou">

#<Nokogiri::XML::Text:0x..fc02b3d48 "Elegant Crested Tinamou"&= gt;

#<Nokogiri::XML::Text:0x..fc02b33f2 = "Little Tinamou">

#<Nokogir= i::XML::Text:0x..fc02b1822 "Slaty-breasted Tinamou">

#<Nokogiri::XML::Text:0x..fc02b1016 "Thi= cket Tinamou">

=3D> 0

Btw, you can also use a more explicit XPath:

'//div[@id=3D"bodyContent"]/div[@id=3D"mw-content-text&q= uot;]/ul[1]/li/a/text()'