"layered" language approach (was: Re: Constructors (was: RE: About KCP and automatic initialize))

Brian T Rice water at tunes.org
Fri Sep 19 04:47:01 UTC 2003


On Fri, 19 Sep 2003, Ian Piumarta wrote:
> Rice:
> > Stef:
> > > Yes the scheme macro for example. I would really like to know what is a
> > > good macro system for a Smalltalk.
> >
> > For reference, Slate has been managing to simplify some language aspects
> > such as is the case for its macro system:
> > Where preceding a selector with ` will result in applying it to its'
> > arguments parse objects
>
> AFAIK, this "quasiquotation" was first *formalised* in Scheme (as part of
> R4RS? around 1985 or so??).  That's one extreme.  At the other extreme it
> has been applied to the C language in the form of "`C" ("tick C" --
> curious how everyone chooses backquote ;) and its compiler "tcc", dating
> from 1995 or so:
>
>    http://www.pdos.lcs.mit.edu/tickc/
>
> In "`C" a backquote preceding a statement or expression delays code
> generation for that entire subtree in the AST until run time, and
> offers selective, per-node replacement within it with values computed
> at runtime.  Semantically, this is identical to quasiquotation
> appearing within the body of a lambda expression in Lisp.

Was there a point to this information? This is rather patronizing and
irrelevant to the point at hand.

` in Slate does NOT correspond to Lisp's backquote. I chose ` because
it has no meaning in Smalltalk syntax; the similarity of use is in a way
coincidental. Its sole purpose is to distinguish macro-sends from normal
message-sends. If you read the manual (the URL I pointed out), you will
see that `quote and `unquote correspond to Lisp's quote/quasiquote and
unquote operators (` or ' and ,).  In Lisp, quotation is a
"non-evaluating" form, transparent to the source text. You can't do this
in Smalltalk, because of the potential conflict with polymorphic sends. Or
rather, we preferred to avoid it since Smalltalkers would find it very
confusing to not be able to immediately distinguish compile-time code from
run-time code, because Smalltalk has a much different role for
compile-time.

I also did not claim to have invented anything but a way to express
syntax-manipulation in a way that's relatively efficient and non-intrusive
for the Smalltalk language and its corresponding mindset. This is what
Stephane asked about, and had it not been phrased this way, I would have
said nothing at all.

> > (which may be evaluated early or template- transformed or
> > directly-manipulated or left alone). Slate has other mechanisms as
> > well for syntax extension, but they are all of this flavor: no
> > special angle-bracket syntax for type declarations or primitive
> > calls are needed.

...

> The usual incentive for "template transformation" type macros is that
> they can be evaluated without having to compile and execute real code
> within the compiler itself at compile time.  (They are friendlier
> toward "static" compilers.)  Many Lisps (and Smalltalk) do not suffer
> from this handicap at all.  Being able to apply quasiquotation within
> a lambda expression (i.e., applying unconstrained rewrites over parts
> of the live AST) to direct the final code generation with runtime
> specialisations is a much simpler, far more general, and conceptually
> transparent replacement for these more "formal", structure-transforming
> macro "minilanguages".

I'm not sure what or whom you're trying to address here. The fact that I
mentioned templates seems to have ignited a rant. There was no special
import to it: it's merely a means of use.

I will add as a note, though, that the act of applying syntax-
manipulating code at run-time is certainly possible in Slate and is one of
the reasons we added it. The simplest version is `evaluate, from the
manual.

> > From water at tunes.org Thu Sep 18 19:13:40 2003
> >
> > In what follows, my answers are phrased in terms of the question which
> > spoke about macros per se, but should be understood as applying to
> > generative or meta-level programming, which has a more understandable
> > relationship to Smalltalk than macros do per se. (For this reason, I found
> > the specific questions to be inappropriate, but still worth answering.)
>
> Macros per se (in any guise) *always* represent some form of
> "generative programming".  They can be *implemented* either as
> meta-level "semantic programs", or as simple rule-based "syntactic
> transformations", or as trivial rule-based structureless "textual
> substitutions".  (One can do very powerful generative programming
> inside the C preprocessor -- yet cpp has no relationship whatsoever
> with meta-level programming.)
>
> So I think the questions were entirely appropriate.

Well, I answered them, so they were fine. I just wanted to point out that
macros are a technical answer, and the question that needs to be asked is
often not directly answered by it. People have argued with me against the
inclusion of macros, without a clear idea of why we added them, or what
role we expect them to fill.

> (Don't forget that people not immersed daily in highly-generative
> and/or meta level-revealing systems -- or who have used only systems
> that deliberatly downplay the meta-level transformations effected by
> user-level "macros" -- might (1) not entirely appreciate that what
> their language calls "macros" are a form of limited exposure to real
> semantic programming on "deep" structures within the implementation,
> or might (2) really be talking about precisely that but lack practical
> experience with a system that presents these features for what they
> are, rather than just labeling them simply as "macros".)
>
> > > (not the technical one
> > > because we are good to access and solve it but the scalability and use
> > > one).
> >
> > The way to deal with or comprehend macros is to grab the output or some
> > intermediate form of it (using full or step-wise macro-expansion.. which
> > is a standard lisp tool) and insert that output into your code. Of course,
> > a good Lisp programmer never does this as such, preferring to write new
> > macros for a refactoring and using the macro-expansions' comparison to
> > debug/unit-test. Or, to write a new common macro that lets you express
> > both the new and old pattern in it, and implement them both on top of
> > that. It's just refactoring and XP applied at the meta-level.
> >
> > One reason that Lisp's tools aren't up to par with Smalltalk's (aside from
> > the big economic collapse) is that the output code of macro-expansion is
> > just a bunch of lists usually,
>
> In Lisp, "just a bunch of lists" *is* an abstract syntax tree, formally
> capable of expressing every language construct, and in which every source
> expression is necessarily (and entirely) expressed.  Lists and atoms and
> lists thereof not only necessary but also sufficient.  This is at the very
> core of the theoretical foundations of Lisp.

If you take "*is*" to mean "is sufficient", then yes, but a syntax tree is
more than just a list: it has properties which are not adequately
expressed as a chain of cons cells of the elements. Just being capable
does not mean being properly expressive. I can express code traversals and
transformations and compiler hints more capably when I use a more apt
structure, as the refactoring browser or even the basic parser do in
Smalltalk.

I'm well aware of many Lisp family variants, the research done in it, and
the research done beyond it. I first used Lisp when I was 12, and have
done extensive research on my own over the last 7-8 years or so, covering
/thousands/ of research papers.

> > so the important meta-information is not explicitly transmitted with
> > it.
>
> The choice of representation of any and all "meta information" rests
> entirely with the language implementor.  If said person chooses to
> represent it in some opaque fashion, unvailable in the meta-level
> exposed to the programmer, then that's not a deficiency inherent in
> "meta-programming" -- it's a deficiency inherent in the implementation
> *design* of that particular language.  Had the designer chosen to make
> the representation of "meta information" correspond either to
> language's syntactic input structures, or to its user-domain data
> structures (in Lisp these are the *same* thing), then there would not
> be any reason whatsoever to lose "meta information".

This seems like a non-response. I'm not arguing that information is
withheld, I'm saying that the user can't get at it when they only have
Lisp's standard Cons-cell representation of syntax trees, whereas if I
have program representation objects to which annotations may be
dynamically added without it implicitly affecting the core meaning, then
information can be retained.  And those objects /are/ user-level objects
in Slate. It just happens that they have a domain orientation towards
syntax, much as the elements of Smalltalk syntax trees, only more flexible
and natural.

By the way, in case you forgot, I /am/ the language implementor. :)

>    [Jim des Rivires and Brian Cantwell Smith, "The implementation
>    of procedurally reflective languages", Proc. ACM Symposium on Lisp
>    and Functional Programming, August 1984.]

I've read it. Basic corpus stuff. I moved past that years ago.

> > At some point, though, you are just talking about generative
> > programming or meta-programming, and you just won't get a magical
> > ability to translate between different custom extensions
>
> Meta-programming is *precisely* all about getting out of the
> user-level box and into the implementation-level box where all the
> interesting customisations are done.  The *extreme* case would be
> this: if the meta-level lets you generate any machine code
> instructions you like in response to seeing some particular syntactic
> input structure, there is no limit on the range of the resulting
> semantics (including infinitely many illegal ones).  Moreover, if the
> correspondance between input structures and generated code is driven by
> rules expressed as "meta-level" structures (defined by those "custom
> extensions") then translating between them is *entirely* possible.

You seem to associate the term meta-programming with anything
implementation-related, whereas I'm just talking about programming at a
level where programs are data, of which implementation meta-programming is
a special case.

> To any sufficiently primitive language design I suppose this *might*
> look like MAGIC.

You're misattributing my target of the term "magic": I was referring to
Stephane's question about how to handle the fact that macros help define
domain-specific or at least case-specific languages, when you have people
sharing code. My point was that you don't get translations for free; you
either restrict the expressiveness or perform extensive pattern-matching
to identify patterns and map them.

> > can know about these effects and handle them well. Dylan's Functional
> > Developer seems to have done the best in the tools area.
>
> Dylan has to have good tools (if only to cope with the heaps of
> irrelevant syntax they're wallowing in ;).

Well, part of the Slate creation rationale was to avoid the complications
of syntax while increasing the level of power.

> > From daniel.a.joyce at worldnet.att.net Thu Sep 18 19:13:54 2003
> >
> > Sounds like Clean/Haskell, in which nearly all features of the language are
> > implemented in the prologue as rewrite rules. At the lowest levels, there are
> > only a few simple operators. Every other language feature is the result of
> > rewrite rules being applied to functions....
>
> Yes!  This is How It Was Meant To Be.  You can probably trace this back
> even further, but the first similar approach I know of was in a Scheme
> system called T (or was it Tea?) and its compiler called Orbit (sometime
> around the mid-80s?).  The language had almost no special forms.  The
> principle was simplicty incarnate: reduce absolutely *everything* to
> lambda expressions, then dedicate 98% of your compiler to optimising the
> hell out of lambdas.

A previous design for Slate was forth/lisp-like in its basic linguistic
simplicity, but never went anywhere because it was too large a step for
people to think in, including myself, and would never attract Forth users.

PS: To explain my initial stance about replies: I much prefer brief
replies, because reading this took me a lot of time, along with composing
an appropriate reply. I generally spend a lot of time on few words, and
gravitate towards those who do the same; it's a character trait or flaw,
depending on your perspective, and you either accept it and work with it,
or implicitly reject it and work against it. My preference is not personal
against anyone.

-- 
Brian T. Rice
LOGOS Research and Development
http://tunes.org/~water/



More information about the Squeak-dev mailing list