From: avi@...4.com (Avi Bryant)
Date: 2002-04-07T06:56:26+09:00
Subject: Re: Is eval a code/design smell?

Sean Middleditch <elanthis@awesomeplay.com> wrote in message news:<1018063188.7769.4.camel@stargrazer>...
> You are missing the point entirely...
> 
> There is nothing wrong with dynamic code.  It's generating code, i.e.
> the actual text passed to a parser and lexer, that I have the problem
> with.

Ok, if you merely dislike the aesthetics of generating text, I
actually agree with you.  I would much rather generate parse trees
directly, if there were a convenient way to do so.  In a prefix-syntax
language, that's the normal way of doing things.  In Ruby, by far the
most convenient way of generating a parse tree is by generating text
and then handing that to a parser, probably the one already provided
by the interpreter.  However, you're absolutely right that if the
facilities existed, manipulating the parse trees directly is a much
better way to go.  But that's not what you're advocating below:

> You can do everything you specified below *without* having to pass code
> to the parser once the application is running - the language should
> provide methods for building classes from components, components which
> would have already been parsed and turned into bytecode/blocks.  I.e.,
> if you have a block, you should be able to add that to a class as a
> method, specifying arguments.

Not all the transformations you might want to apply on a class can be
expressed in this kind of black box way.  Yes, it's a very clean,
useful, and powerful way of doing things - in fact, after I requested
it, matz did add the ability to use a block to specify a method
(define_method, which has different semantics in recent versions of
Ruby than it used to).  But it's simply not as powerful as direct
parse tree manipulation.

The one environment that comes close enough to being powerful enough,
IMO, that code generation is unnecessary, is Smalltalk-80 or its close
descendants (Squeak, VisualWorks).  That's because you have direct
access to virtually everything in the environment: parser, compiler,
bytecode, stack frames, etc.  One of my favorite tricks is that you
can do the equivalent of this:

class MyClass
  def self.compilerClass
     return MyCompiler
  end

  ...
end

And all the instance methods of MyClass will be compiled using
MyCompiler.  This can be used for instrumentation (ie, adding
persistence code around inst var accesses), or for embedding entirely
new languages - one Smalltalk parser-generator uses this to allow
methods written as EBNF rules.

Classes in Smalltalk contain hash tables of method objects, which are
just arrays of bytecode - do a lookup, mess with the bytecode, and the
method changes.  Is this the kind of language support you're thinking
of?

But even in Smalltalk I wish I had macros - not because they make
anything more possible, just that they would make some things a lot
easier.

> Again, nothing you said requires an eval(), requires passing a string
> into a parser to generate new blocks... all the actual, written code
> should be already parsed - just in need of "assembling" (not in the
> machine code fashion) into the proper form.

Early versions of ELIDE worked like this - all the code in the
"macros" was pre parsed, and at runtime (ie, when the orginal code was
being transformed), all the operations were purely on parse trees. 
Although this was more efficient, it turned out to be a major
headache, and we ended up chucking the partial evaluation and just
reparsed everything as we needed to.  Now, this was much different
from eval() - you were basically calling parse(), inserting the result
into an exisiting parse tree, and then dumping the code out at the end
to be compiled, so there weren't any safety concerns.  But the thing
is that for non-trivial operations on non-trivial syntax, plain
textual transformations are the easiest for humans to deal with.  And
I'm not convinced that parse tree manipulation is any inherently safer
than textual manipulation - it's more restricted, yes, but I doubt
it's restrictive enough to avoid exploits.