From: "A. S. Bradbury" <asbradbury@...>
Date: 2006-05-25T05:54:46+09:00
Subject: Re: [ANN] Recipients of Google Summer of Code awards

On Wednesday 24 May 2006 16:47, Victor Shepelev wrote:
> Are somewhere detailed description for all those projects? Some sound a bit
> confusingly.
> Victor.

My snappily titled proposal "Automated Wrapper Generation for Information 
Extraction" has been accepted. I hope you'll find it's a lot more interesting 
than it might sound. A potentially suitable description would be "a web 
scraper that writes itself". Tools like RubyfulSoup go a long way towards 
making it easy to extract data from the web, but I believe in many cases we 
can go one step further. My library will take labeled examples, and generate 
extraction rules.

This is actually a topic that has received quite a lot of attention in 
academic circles (search for some of the keywords from my proposal title). 
Despite the existence of several excellent papers detailing useful and highly 
successful approaches to the problem, the FOSS community doesn't seem to have 
any libraries that make it easy to extract data in this way. Clearly, I'm 
hoping to harness existing research and apply it to improve that situation. 
I'll write detailed documentation of my approach later on, it's important 
people can understand what's going on behind the scenes. For now, my first 
problem is what to name the project. Has anyone any smart ideas?

What would you call a wrapper generator/information extractor written in Ruby? 
the name shouldn't indicate the library is (x)html/web-specific (it won't be, 
although I think that is one of the most compelling use cases), and anything 
about wrapper generation is probably going to make most people think of SWIG.

Alex