From: Robert Klemme <lists@...>
Date: 2012-04-05T00:07:12+09:00
Subject: Re: RegEx for a string between two symbols

Jan E. wrote in post #1054964:
> Robert Klemme wrote in post #1054952:
>>> ####
>>> dictionary = {}
>>> File.foreach 'C:/dictionary.txt' do |line|
>>>   next if line.strip.empty?             # skip empty lines
>>>   parts = line.split('::').map &:strip  # split line at the double colon
>>
>> That's unsafe because it will break if there is a '::' in the
>> definition.
>
> It's meant to break, because multiple double colons make the line
> ambiguous (unless you make certain implications about the words and
> definitions).

Since we are talking about "words" I deem it reasonable to assume they
do not contain "::".

> If you actually want to allow literal double colons in a safe way, you'd
> have to implement some kind of escape syntax. But I don't think that's
> necessary.

Exactly, because words do not contain "::". :-)  I think this is a
pretty low restriction and we can then use a slightly modified regexp by
substituting "\w+" with "\S.*?":

%r{\A\s*(\S.*?)\s*::\s*+(.*?)\s*\z}

Now there needs to be at least one "::" in the line and the first "::"
on the line is the delimiter.  No escaping needed.  And the regexp does
the stripping already.

>>     if %r{\A\s*(\w+)\s*::\s*+(.*?)\s*\z} =~ line
>>       yield $1, $2 # word, def
>
> I find the \w+ pattern much too strict for words in a dictionary, since
> they may include spaces, hyphens or even special characters like
> umlauts.

Let's ask the OP: Christopher, what is a "word" in your application?  Do
word characters (\w) suffice or are you using a more broad definition of
"word".

>> Btw, I wouldn't use symbols for the words because there are potentially
>> many and they are not GC'ed.
>
> If the word list actually gets *that* big, you're better off using a
> database, anyway.

Maybe.  But since Christopher appears to start learning Ruby I think he
should start using Symbol and String properly right away which will
avoid hassle later on.  Since symbols are never GC'ed most of the time
it is a bad idea IMHO to create them dynamically, i.e. from input.
Symbols are best used for a limited number of identifiers used
throughout the program (e.g. names of states).  Everything which comes
from user input is usually better off being placed in Strings.

Kind regards

robert

-- 
Posted via http://www.ruby-forum.com/.