From: phlip_cpp@... (Phlip)
Date: 2001-10-26T06:34:54+09:00
Subject: [ruby-talk:23359] Re: Fuzzy string match upgrade needed

Bill Kelly wrote:

> Tokenizer = /-?\d+(?:\.\d+)?|[a-zA-Z]+|(?:-(?!\d)|[^\d\s-])/

Thanks! Now for a tweak...

Tokenizer = /-?\d+(?:\.\d+)(?:e(?:-)\d+)?|[a-zA-Z]+|(?:-(?!\d)|[^\d\s-])/
"1 + -8.71 -8.1e5 + 8.1e-5 + -8.1e-5".scan(Tokenizer)

That yields this:

["+", "-8.71", "-8.1", "e", "+", "8.1e-5", "+", "-8.1e-5"]

I tried to add support for exponentials, like the 8.1e-5.

But notice my first exponential, -8.1e5, comes out broken into two
elements, with the 5 missing.

Two questions. How do we polish this? and how do we write Tokenizer
as, say, three little expressions (with intent-revealing names) that
we can then concatenate into one big expression before use?

-- 
Phlip

http://www.greencheese.org/HoneySatellite