From: Aaron Patterson <aaron@...>
Date: 2009-11-24T04:59:40+09:00
Subject: Re: Nokogiri help

On Thu, Nov 19, 2009 at 01:54:47PM +0900, Jeremy Woertink wrote:
> I checked out the page response, and this is what I got back
> 
> >> page.response
> => {"cache-control"=>"private", "connection"=>"close", 
> "p3p"=>"policyref=\"http://p3p.yahoo.com/w3c/p3p.xml\", CP=\"CAO DSP COR 
> CUR ADM DEV TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi 
> PUBi IND PHY ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE GOV\"", 
> "date"=>"Thu, 19 Nov 2009 04:39:19 GMT", "content-type"=>"text/html", 
> "content-encoding"=>"gzip", "set-cookie"=>"B=c4mf1f55g9ivn&b=3&s=9v; 
> expires=Tue, 02-Jun-2037 20:00:00 GMT; path=/; domain=.yahoo.com"}
> 
> 
> so, where content-encoding is gzip, is this what "should" be UTF-8?

No.  That means they are just not specifying a character encoding?  Was
there one in the HTML document itself?

> I just updated my libxml2 as well so I'm using libxml2 @2.7.3_0 
> (active). Is there an attribute I can set somewhere that will allow me 
> to parse the page using the gzip encoding?

No.  It should be unzipped before sending to the parser.

> Thanks for the help man!

No problem.  :-)

-- 
Aaron Patterson
http://tenderlovemaking.com/