From: dblack@...
Date: 2006-11-29T09:40:44+09:00
Subject: Re: "yield" and "old-way iteration"

HI --

On Wed, 29 Nov 2006, Bjarke Walling wrote:

> Hi,
>
> I am new to Ruby, but I find the language easy to learn and use.
> However I have some code that I think could be written smarter.
>
> I am writing a lexer and parser for a small language I have created.
> The first part splitting some input into tokens was easy to write using
> "yield" (20 lines or so). I was actually a little overwhelmed how easy
> it was. The next part is to examine these tokens and parse them into
> language structures. I want to create a class with "current" and "next"
> methods to get the current token and fetch the next (advance the
> pointer). I have solved it by using the first lexer yielding tokens and
> collecting them in an array. Afterwards I can fetch tokens from the
> array. But could it be done in a smarter way?
>
> My code is like this:
>
> class FirstParse
>  def initialize
>    ...
>  end
>  def each
>    ... yield tokens ...
>  end
> end
>
> class SecondParse
>  def initialize
>    @tokens = []
>    @index = 0
>    first_parse = FirstParse.new
>    for token in first_parse
>      @tokens.add token
>    end
>  end
>  def current
>    @tokens[@index]
>  end
>  def next
>    @index++

That won't parse :-)  There's no ++ operator in Ruby; you'll want to
do:
   @index += 1

>    self.current
>  end
>  def read_structure1
>    ... read structures ...
>  end
>  def read_structure2
>    ... read structures ...
>  end
> end
>
> Am I being to "Java'ish" or what do you think. It is not a big problem
> since the code works, but do I really need to load the tokens into an
> array first?

I would say that if you're going to load the tokens into an array,
don't do it by yielding; do it by returning an array.  (All the code
that follows is very sketchy and just intended to illustrate the broad
picture.)

   class FirstParse
     attr_reader :tokens
     def initialize
       # put all tokens in @tokens array
     end
   end

   class SecondParse
     def initialize
       @tokens = FirstParse.new.tokens
     end
   end

If you want to, you can just yield from the first parse to the second
parse.

   class FirstParse
     def initialize
       while (# get token from stream)
         yield token
       end
     end
   end

   class SecondParse
     def initialize
       FirstParse.new.each do |token|
         # do something with token here -- don't save it
       end
     end
   end

That way you don't have to maintain your own pointer (though you could
add it back in if needed it for some other reason).


David

-- 
                   David A. Black | dblack@wobblini.net
Author of "Ruby for Rails"   [1] | Ruby/Rails training & consultancy [3]
DABlog (DAB's Weblog)        [2] | Co-director, Ruby Central, Inc.   [4]
[1] http://www.manning.com/black | [3] http://www.rubypowerandlight.com
[2] http://dablog.rubypal.com    | [4] http://www.rubycentral.org