From: Peter Hickman <peterhickman386@...>
Date: 2010-12-13T19:47:32+09:00
Subject: Re: scraping but failed due to the uniqueness of the site

On 13 December 2010 10:08, Soichi Ishida
<soujiro0725+rubyforum@gmail.com> wrote:
> I am guessing that this Ruby code activates JavaScript or whatever to
> manipulate the video.

Given that you are only using net/http I would say that the javascript
is not being run at all.

This code is a step in the right direction:

require 'rubygems'

require 'open-uri'
require 'nokogiri'

URL="http://voxaleadnews.labs.exalead.com/play.php?flv=MSNBC_Brian_Williams/pdv_nn_netcast_m4v-12-12-2010-173721.mp4&q=&language=en"

html = open(URL).read
doc  = Nokogiri::HTML(html)

doc.search("//div[@id='fulltext']//span").each do |span|
puts span
end