From: Dominik Bathon <dbatml@...>
Date: 2006-04-07T05:36:30+09:00
Subject: Re: String generalization

On Thu, 06 Apr 2006 19:16:44 +0200, Robert Dober <robert.dober@gmail.com>  
wrote:

> On 4/6/06, Dominik Bathon <dbatml@gmx.de> wrote:
>>
>> [SNIP]
>>
>>
>> The part that finds the repeating patterns is a bit ugly, but it should
>> work. Here are the results:
>>
>> $ ruby generalize_rx.rb
>> 12345 => /\d+/
>> ABCDE => /[A-Z]+/
>> john.smith@my.super.server.rb => /[a-z]+\.[a-z]+@([a-z]+\.)+[a-z]+/
>> 123-45-678-90 => /(\d+\-)+\d+/
>> item: => /[a-z]+:/
>> Peter, SZINEK => /\w+,\ [A-Z]+/
>> http://www.google.com => /[a-z]+:\/\/([a-z]+\.)+[a-z]+/
>> jd�;d:L348kddd3 => /[a-z]+\243;[a-z]:\w+/
>
>
> Well that is quite impressive, but I could not figure out why
> "12a".you_know_the_name_of_the_method gives /\w+/, it clearly should give
> /\d+[a-z]+/, no?

split(/(\W+)/) splits the string into alphanumerical and  
non-alphanumerical parts:

irb(main):001:0> "12a".split(/(\W+)/)
=> ["12a"]
irb(main):002:0> ".12a.iu-23..4234".split(/(\W+)/)
=> ["", ".", "12a", ".", "iu", "-", "23", "..", "4234"]

Then my implementation escapes the non-alphanumeric parts and tries to  
find a pattern for the alphanumeric parts (from PATTERNS_TO_GENERALIZE).
The first pattern that matches "12a" is /\w+/ so it gets choosen.

If you want /\d+[a-z]+/ you can either add more patterns to  
PATTERNS_TO_GENERALIZE, or try to further split the alphanumeric parts.

But what would you do with "1b23xy3sa"? Should that be /\w+/ or  
/\d[a-z]\d+[a-z]+\d[a-z]+/?


Dominik