From: Glenn Gillen <glenn.gillen@...>
Date: 2007-09-22T00:45:30+09:00
Subject: Re: Screen Scraping Advice


On 20/09/2007, at 7:54 PM, brabuhr@gmail.com wrote:

>>> give me advice? Should I just do keyword searches for those 20+
>>> products? Or is there a better approach?
>
> On 9/20/07, Glenn Gillen <glenn.gillen@gmail.com> wrote:
>> I'm slightly biased, but scrubyt should be able to do most of the
>> remaining heavy lifting for you
>>
>> http://scrubyt.org/
>
> On that note:
>
> <snip>
> (tons of improvement needed, but):
> <snip>

It's by no means a silver bullet, but could very well get you 80%  
there. Setup a basic learning extract that is fairly generic looking  
for terms you know will exist on the domains you want (say a model  
number and a dollar sign?), have it loop over the URLs with products  
on them, output the learner to production extractor and then tweak  
the sites that aren't giving you the exact results you want.

Or, make life easier if you can and let froogle put it all into a  
single format for you.

Best of luck,

Glenn