From: Heesob Park <phasis@...>
Date: 2009-05-21T16:35:30+09:00
Subject: Re: html method on watir don't return the real html

Hi,

2009/5/19 Mario Ruiz <tcblues@gmail.com>:
> Hi,
> I'm using watir to get the html of a page in order to verify every
> single page on w3c.org but what I get with watir is something like:
> <HTML lang=is xml:lang="is"
> xmlns="http://www.w3.org/1999/xhtml"><HEAD><TITLE>Certus Games</TITLE>
> <META http-equiv=Cache-Control content=no-cache>
> <META http-equiv=Pragma content=no-cache>
> <META http-equiv=Expires content=-1>
> <META http-equiv=Content-Type content="text/html; charset=utf-8"><LINK
> media=screen href="/CF/css/screen.css" type=text/css
> rel=stylesheet><LINK media=print href="/CF/css/print.css" type=text/css
> rel=stylesheet><LINK media=screen href="/CF/css/lib/iestyles.css"
> type=text/css rel=stylesheet>
> <SCRIPT src="/CF/js/lib/jquery.js" type=text/javascript></SCRIPT>
> ....
>
> And the real content is:
>
> <?xml version="1.0" encoding="UTF-8" ?>
> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
> "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
>
> <html xmlns="http://www.w3.org/1999/xhtml" lang="is" xml:lang="is">
>  <head>
>    <meta http-equiv="Cache-Control" content="no-cache"/>
>    <meta http-equiv="Pragma" content="no-cache"/>
>    <meta http-equiv="Expires" content="-1"/>
>    <meta http-equiv="Content-Type" content="text/html; charset=utf-8"
> />
> ....
>
>
> Any idea how to get the real html content?
>
If you are using Ruby 1.8.6 on Windows, try this code:

require 'win32/api'
require 'watir'

IID_IPersistStreamInit =
[0x7FD52380,0x4E07,0x101B,0xAE,0x2D,0x08,0x00,0x2B,0x2E,0xC7,0x13].pack('LSSC8')

CreateStreamOnHGlobal = Win32::API.new('CreateStreamOnHGlobal', 'LLP',
'L','ole32')
OleSaveToStream = Win32::API.new('OleSaveToStream', 'LL', 'L','ole32')
GetHGlobalFromStream = Win32::API.new('GetHGlobalFromStream', 'LP', 'L','ole32')
GlobalSize = Win32::API.new('GlobalSize', 'L', 'L')
GlobalLock = Win32::API.new('GlobalLock', 'L', 'L')
CopyMemory = Win32::API.new('RtlMoveMemory', 'PLL', 'V')
GlobalUnlock = Win32::API.new('GlobalUnlock', 'L', 'L')

browser = Watir::Browser.new
browser.goto("http://www.google.com")

ptr = browser.document.inspect.scan(/:(0x[\da-f]+)?>/).to_s.hex
p = 0.chr * 4
CopyMemory.call(p,ptr+16,4)
data = p.unpack('L').first
p = 0.chr * 4
CopyMemory.call(p,data,4)
dispatch = p.unpack('L').first

lpVtbl = 0.chr * 4
table = 0.chr * 28
CopyMemory.call(lpVtbl,dispatch,4)
CopyMemory.call(table,lpVtbl.unpack('L').first,28)
table = table.unpack('L*')
queryInterface = Win32::API::Function.new(table[0],'PPP','L')
p = 0.chr * 4
hr = queryInterface.call(dispatch,IID_IPersistStreamInit,p)
persiststream = p.unpack('L').first

p = 0.chr * 4
CreateStreamOnHGlobal.call(0, 1, p)
stream = p.unpack('L').first
if OleSaveToStream.call(persiststream,stream) == 0
	p = 0.chr * 4
	GetHGlobalFromStream.call(stream,p)
	h = p.unpack('L').first
	size = GlobalSize.call(h)
	buf = 0.chr * size
	ptr = GlobalLock.call(h)
	if ptr != 0
		CopyMemory.call(buf,ptr,size)
		GlobalUnlock.call(ptr)
		html = buf[16..-1]
	else
		html = nil
	end
end

puts html


Regards,
Park Heesob