6 Modules and Bindings

As an embedded domain-specific language, Scribble follows a long tradition of using Lisp- and Scheme-style macros to implement little languages. In particular, Scribble relies heavily on the Scheme notion of syntax objects (Sperber 2007), which are fragments of code that have lexical-binding information attached. Besides using syntax objects in the usual way to implement macros, Scribble uses syntax objects to carry lexical information all the way through document rendering. For example, @scheme[lambda] expands to roughly (typeset-id #'lambda), where #'lambda is similar to 'lambda but produces a syntax object (with its lexical information intact) instead of a symbol.

At the same time, many details of Scribble’s implementation rely on PLT Scheme extensions to Scheme macros. Continuing the above example, the typeset-id function applies PLT Scheme’s identifier-label-binding function to the given syntax object to determine the source module of its binding. The typeset-id function can then construct a cross-reference key based on the identifier and the source module; the documentation for the binding pairs the same identifier and source module to define the target of the cross-reference.

A deeper dependence of Scribble on PLT Scheme relates to #lang parsing. The #lang notation organizes reader extensions of Scheme (i.e., changes to the way that raw text is converted to S-expressions) to allow new forms of surface syntax. The identifier after #lang in the original source act as the “language” of a module.

The module form is PLT Scheme’s core module form, and it generalizes the standard library form (Sperber 2007) to give macros more control in transforming the body of a module. Within a module, the first identifier is the relative name of the module, and the second identifier indicates a module to supply initial bindings for the module body. In particular, the initial import of a module is responsible for supplying a #%module-begin macro that is implicitly applied to the entire content of the module.

In the case of scribble/doclang, the #%module-begin macro lifts out all import and definitions forms in the body, passes all remaining content to the decode function, and binds the result to an exported doc identifier. Thus, macro expansion converts the hello module to the following:

(module hello scheme/base

(require scribble/doclang

scribble/manual)

(provide doc)

(define doc

(decode

"\n"  "\n"

"It was a " (bold "dark") " and "

(italic "stormy") "night." "\n")))

A subtlety in the process of lifting out import and definition forms is that they might not appear directly, but instead appear in the process of macro expansion. For example, include-section expands to a require of the included document plus a reference to the document. The #%module-begin macro of scribble/doclang therefore relies on a PLT Scheme facility for forcing the expansion of sub-forms. Specifically, #%module-begin uses local-expand to expand each sub-form just far enough to determine whether it is an import form, definition form, or expression. If the sub-form is an import or definition, then #%module-begin suspends further work and lifts out the import or definition immediately; the import or definition can then supply bindings for further expansion of the module body. The need to suspend and continue lifting explains the () inserted in the body of a module by the scribble/doc reader; #%module-begin uses that position to track the sub-forms that have been expanded already to expressions.

Aside from (1) the ability to force the expansion of nested forms and (2) the ability of macros to expand into new imports, macro expansion of a module body is essentially the same as for libraries in the current Scheme standard (Sperber 2007). Where the standard allows choice in the separation of phases, we have chosen maximal separation in PLT Scheme, so that compilation and expansion as consistent as possible (Flatt 2002). That is, bindings and module instantiations needed during the compilation of a module are kept separate from the bindings and instantiations needed when executing a module for rendering.

Furthermore, to support the connection between documentation and library bindings, PLT Scheme introduces a new phase that is orthogonal to compile time or run time: the label phase level. As noted in Scribbling Code, a for-label import introduces bindings for documentation without triggering the execution of the imported module. In PLT Scheme, the same identifier can have different bindings in different phases. For example, when documenting the Intermediate Scheme pedagogical language, a document author would like uses of lambda to link to the lambda specification for Intermediate Scheme, while procedures used to implement the document itself will more likely use the full PLT Scheme language, which is a different lambda. The two different uses of lambda are kept straight naturally and automatically by separate bindings in separate phases.