s48-refman/module-sys/macros.texi

@node Macros in concert with modules
@section Macros in concert with modules

One reason that the standard Scheme language does not support a module
system yet is the issue of macros and modularity.  There are several
issues to deal with:

@itemize @bullet
@cindex separate compilation
@item
that compilation of code that uses macros requires presence of those
macros' definitions, which prevents true separate compilation, because
those macros may be from other modules;

@cindex hygiene of macros in modules
@cindex referential transparency of macros in modules
@cindex macro hygiene in modules
@cindex macro referential transparency in modules
@item
that a macro's expansion must preserve referential transparency and
hygiene, for example in cases where it refers to names from within the
module in which it was defined, even if those names weren't exported;
and

@cindex phase separation
@cindex towers of evaluation phases
@item
that a macro's code may be arbitrary Scheme code, which in turn can use
other modules, so one module's compile-time, when macros are expanded,
is another's run-time, when the code used in macros is executed by the
expander: this makes a tower of phases of code evaluation over which
some coherent control must be provided.
@end itemize

@noindent
Scheme48's module system tries to address all of these issues
coherently and comprehensively.  Although it cannot offer @emph{total}
separate compilation, it can offer incremental compilation, and
compiled modules can be dumped to the file system & restored in the
process of incremental compilation.@footnote{While such facilities are
not built-in to Scheme48, there is a package to do this, which will
probably be integrated at some point soon into Scheme48.}

Scheme48's module system is also very careful to preserve non-local
module references from a macro's expansion.  Macros in Scheme48 are
required to perform hygienic renaming in order for this preservation,
however; @pxref{Explicit renaming macros}.  For a brief example,
consider the @code{delay} syntax for lazy evaluation.  It expands to a
simple procedure call:

@lisp
(delay @var{expression})
  @expansion{} (make-promise (lambda () @var{expression}))@end lisp

@noindent
However, @code{make-promise} is not exported from the @code{scheme}
structure.  The expansion works correctly due to the hygienic renaming
performed by the @code{delay} macro transformer: when it hygienically
renames @code{make-promise}, the output contains not the symbol but a
special token that refers exactly to the binding of @code{make-promise}
from the environment in which the @code{delay} macro transformer was
defined.  Special care is taken to preserve this information.  Had
@code{delay} expanded to a simple S-expression with simple symbols, it
would have generated a free reference to @code{make-promise}, which
would cause run-time undefined variable errors, or, if the module in
which @code{delay} was used had its @emph{own} binding of or imported a
binding of the name @code{make-promise}, @code{delay}'s expansion
would refer to the wrong binding, and there could potentially be
drastic and entirely unintended impact upon its semantics.

@cindex reflective tower
@cindex syntactic tower
@cindex @code{for-syntax}
Finally, Scheme48's module system has a special design for the tower of
phases, called a @dfn{reflective tower}.@footnote{This would be more
accurately named `syntactic tower,' as it has nothing to do with
reflection.}  Every storey represents the environment available at
successive macro levels.  That is, when the right-hand side of a macro
definition or binding is evaluated in an environment, the next storey
in that environment's reflective tower is used to evaluate that macro
binding.  For example, in this code, there are two storeys used in the
tower:

@lisp
(define (foo ...bar...)
  (let-syntax ((baz ...quux...))
    ...zot...))@end lisp

@noindent
In order to evaluate code in one storey of the reflective tower, it is
necessary to expand all macros first.  Most of the code in this example
will eventually be evaluated in the first storey of the reflective
tower (assuming it is an ordinary top-level definition), but, in order
to expand macros in that code, the @code{let-syntax} must be expanded.
This causes @code{...quux...} to be evaluated in the @emph{second}
storey of the tower, after which macro expansion can proceed, and long
after which the enclosing program can be evaluated.

@cindex @code{for-syntax}
The module system provides a simple way to manipulate the reflective
tower.  There is a package clause, @code{for-syntax}, that simply
contains package clauses for the next storey in the tower.  For
example, a package with the following clauses:

@lisp
(open scheme foo bar)
(for-syntax (open scheme baz quux))@end lisp

@noindent
has all the bindings of @code{scheme}, @code{foo}, & @code{bar}, at the
ground storey; and the environment in which macros' definitions are
evaluated provides everything from @code{scheme}, @code{baz}, &
@code{quux}.

With no @code{for-syntax} clauses, the @code{scheme} structure is
implicitly opened; however, if there are @code{for-syntax} clauses,
@code{scheme} must be explicitly opened.@footnote{This is actually only
in the default config package of the default development environment.
The full mechanism is very general.}  Also, @code{for-syntax} clauses
may be arbitrarily nested: reflective towers are theoretically infinite
in height.  (They are internally implemented lazily, so they grow
exactly as high as they need to be.)

Here is a simple, though contrived, example of using @code{for-syntax}.
The @code{while-loops} structure exports @code{while}, a macro similar
to C's @code{while} loop.  @code{While}'s transformer unhygienically
binds the name @code{exit} to a procedure that exits from the loop.
It necessarily, therefore, uses @embedref{Explicit renaming macros,
explicit renaming macros} in order to break hygiene; it also, in the
macro transformer, uses the @code{destructure} macro to destructure the
input form (@pxref{Library utilities}, in particular, the structure
@code{destructuring} for destructuring S-expressions).

@lisp
(define-structure while-loops (export while)
  (open scheme)
  (for-syntax (open scheme destructuring))
  (begin
    (define-syntax while
      (lambda (form r compare)
        (destructure (((WHILE test . body) form))
          `(,(r 'CALL-WITH-CURRENT-CONTINUATION)
             (,(r 'LAMBDA) (EXIT)
               (,(r 'LET) (r 'LOOP) ()
                 (,(r 'IF) ,test
                   (,(r 'BEGIN)
                     ,@@body
                     (,(r 'LOOP)))))))))
      (CALL-WITH-CURRENT-CONTINUATION LAMBDA LET IF BEGIN))))@end lisp

This next @code{while-example} structure defines an example procedure
@code{foo} that uses @code{while}.  Since @code{while-example} has no
macro definitions, there is no need for any @code{for-syntax} clauses;
it imports @code{while} from the @code{while-loops} structure only at
the ground storey, because it has no macro bindings to evaluate the
transformer expressions of:

@lisp
(define-structure while-example (export foo)
  (open scheme while-loops)
  (begin
    (define (foo x)
      (while (> x 9)
        (if (integer? (sqrt x))
            (exit (expt x 2))
            (set! x (- x 1)))))))@end lisp