From: "Brian Schröder" <ruby@...>
Date: 2004-10-25T18:20:41+09:00
Subject: Re: parsing html

If you don't mind senseless words like "img" that come from html markup:

--8<---
require 'open-uri' open('http://ruby.brian-schroeder.de').read.scan(/[-\w����]+/i)
--8<---

If you have valid xhtml:

--8<---
require 'rexml/document'
require 'open-uri'

include REXML 
Document.new(open('http://ruby.brian-schroeder.de')).
  elements.to_a('//').
  map{|e| e.texts.map{|t|t.value} }.
  join(' ').
  scan(/[-\w����]+/i).
  sort.
  uniq
--8<---

hth,

Brian

PS: I'm shure the text-extraction with rexml can be done in a nicer/more efficent way.



On Mon, 25 Oct 2004 02:39:08 +0900
Martin Pfeffer <udlduz@chello.at> wrote:

> hi
> my problem is i need a file with german words and so i try to create a 
> file parsing html sites and write extracted words to a database so my 
> questizn is what is the easyest way to extract text from html pages?
> thx
> Martin
>