For Relax NG, annotations take the form of additions of elements and attributes from other namespaces in Relax NG schemas. When we have seen in the previous chapter "Chapter 12: Writing extensible schemas" how to deal with extensibility for our schemas and instance documents, we've been relying on elements and attributes which syntax and semantic is precisely defined in the Relax NG specification. The annotations which we will see in this chapter are also a form of extensibility, but an extensibility of the Relax NG vocabulary itself.

The scope of applications based on annotations is as wide as our imagination! However, there are common trends in the existing usage of schema annotation and we can distinguish between annotations for documentation purposes and annotations for applications.  In this second category we can further distinguish between pre-processing annotations to generate a variety of schemas from a common one, annotations for helping to generate something out of a Relax NG schema and annotations extending the features of Relax NG. But, before walking through these applications of schema annotations, we need to see the syntax for embedding annotations within Relax NG schemas.

!!!Common principles for annotating Relax NG schemas

Instead of defining specific elements and attributes reserved for annotations, Relax NG has opened its language to allow foreign attributes (i.e. attributes from any namespace other than the Relax NG namespace) in all its elements and to allow elements from either no namespace or from any namespace other than the Relax NG namespace in all its elements with a content model which is empty or element only (i.e. all its elements except "value" and "param" which have a text only content model). Relax NG is thus strictly following the principle of open schema which we've presented in the last chapter.

!!Annotation using the XML syntax

This is both easy and flexible, at least with the XML syntax and it is very straightforward to add annotations using foreign elements, for instance:

 <?xml version="1.0" encoding="utf-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0" xmlns:dc="http://purl.org/dc/elements/1.1/">
   <dc:title>Relax NG flat schema for our library</dc:title>
   <dc:author>Eric van der Vlist</dc:author>
   <start>
     <element name="library">
       <oneOrMore>
         <ref name="book-element"/>
       </oneOrMore>
     </element>
   </start>
   .../...
 </grammar>

or:

 <?xml version="1.0" encoding="utf-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0" xmlns:xhtml="http://www.w3.org/1999/xhtml">
   <xhtml:div>
     <xhtml:h1>Relax NG flat schema for our library</xhtml:h1>
     <xhtml:p>This schema has been written by
              <xhtml:a href="http://dyomedea.com/vdv">Eric van der Vlist</xhtml:a>.</xhtml:p>
   </xhtml:div>
   .../...
 </grammar>

or using foreign attributes:

 <?xml version="1.0" encoding="utf-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0"
          xmlns:xlink="http://www.w3.org/1999/xlink">
   <start>
     <element name="library"
       xlink:type="simple"
       xlink:role="http://www.w3.org/1999/xhtml"
       xlink:arcrole="http://www.rddl.org/purposes#reference"
       xlink:href="library.xhtml">
       <oneOrMore>
         <ref name="book-element"/>
       </oneOrMore>
     </element>
   </start>
   .../...
  </grammar>

!!Annotations using the compact syntax

Annotations are much more challenging for the compact syntax which, not being XML had no built-in support for this kind of extensibility and an alternative syntax based on square brackets ("[[" and "]") has been developed to embed XML structures. Unfortunately, this isn't playing very well with the other constructions used in the compact syntax and, if the syntax to define the annotations is consistent, the syntax to include them within a schema is slightly different according to the location in the schema.

Without being very difficult, annotations using the compact syntax may have a strange looking and are easily error prone. Translating between the compact and the XML syntax is very easy using tools such as "Trang" and you may feel safer if you always convert to the XML syntax to edit your annotations. Anyway, let's take a look at this weird syntax.

!Grammar annotations

The easiest annotations to write are foreign elements in a grammar element. These annotations and called "grammar annotations" and correspond to the two first examples given with the XML syntax.

 <?xml version="1.0" encoding="utf-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0" xmlns:dc="http://purl.org/dc/elements/1.1/">
   <dc:title>Relax NG flat schema for our library</dc:title>
   <dc:author>Eric van der Vlist</dc:author>
   <start>
     <element name="library">
       <oneOrMore>
         <ref name="book-element"/>
       </oneOrMore>
     </element>
   </start>
   .../...
 </grammar>

would be written:

 namespace dc = "http://purl.org/dc/elements/1.1/"

 dc:title [[ "Relax NG flat schema for our library" ]

 dc:author [[ "Eric van der Vlist" ]

 start = element library { book-element+ }

Note how the annotation has been included by using its qualified name ("dc:title" or "dc:author"). This piece is specific to grammar annotations while the syntax "[[ element content ]" used to represent its content is more generic.

These annotations can have structured contents with children elements and attributes:

 <?xml version="1.0" encoding="utf-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0" xmlns:xhtml="http://www.w3.org/1999/xhtml">
   <xhtml:div>
     <xhtml:h1>Relax NG flat schema for our library</xhtml:h1>
     <xhtml:p>This schema has been written by <xhtml:a href="http://dyomedea.com/vdv">Eric van der Vlist</xhtml:a>.</xhtml:p>
   </xhtml:div>
   .../...
 </grammar>

or, using the compact syntax:

 namespace xhtml = "http://www.w3.org/1999/xhtml"

 xhtml:div
 [[
    xhtml:h1 [[ "Relax NG flat schema for our library" ]
    xhtml:p
    [[
       "This schema has been written by "
       xhtml:a [[ href = "http://dyomedea.com/vdv" "Eric van der Vlist" ]
       "."
    ]
 ]

 start = element library { book-element+ }
 .../...

Note how, within the annotation, the same principles have been applied recursively and how the "href" attribute has been expressed as 'href = "http://dyomedea.com/vdv"'.

These grammar annotations are always foreign elements and another mechanism (the so called "initial annotations") would be used to express annotations through foreign attributes.

!Initial annotations

Initial annotations are used to define annotations (through foreign elements or attributes) which will be appended as the first children of the next pattern. This is the option we must always use to define annotations as foreign attributes, such as in:

 <?xml version="1.0" encoding="utf-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0"
          xmlns:xlink="http://www.w3.org/1999/xlink">
   <start>
     <element name="library"
       xlink:type="simple"
       xlink:role="http://www.w3.org/1999/xhtml"
       xlink:arcrole="http://www.rddl.org/purposes#reference"
       xlink:href="library.xhtml">
       <oneOrMore>
         <ref name="book-element"/>
       </oneOrMore>
     </element>
   </start>
   .../...
  </grammar>

which would be written:

 namespace xlink = "http://www.w3.org/1999/xlink"

 start =
    [[
       xlink:type = "simple"
       xlink:role = "http://www.w3.org/1999/xhtml"
       xlink:arcrole = "http://www.rddl.org/purposes#reference"
       xlink:href = "library.xhtml"
    ]
    element library { book-element+ }

Note how the foreign elements have been wrapped within square brackets and also that the annotations are not included in the "element" pattern but are preceding it. These syntax with square brackets wrapping annotations without a preceding name is what makes an "initial annotation". This is not specific to attributes and elements or both and attributes could have been included as initial annotations:

 <?xml version="1.0" encoding="utf-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0"
          xmlns:xlink="http://www.w3.org/1999/xlink"
          xmlns:dc="http://purl.org/dc/elements/1.1/">
   <start>
     <element name="library"
       xlink:type="simple"
       xlink:role="http://www.w3.org/1999/xhtml"
       xlink:arcrole="http://www.rddl.org/purposes#reference"
       xlink:href="library.xhtml">
       <dc:title>The library element</dc:title>
       <dc:author>Eric van der Vlist</dc:author>
       <oneOrMore>
         <ref name="book-element"/>
       </oneOrMore>
     </element>
   </start>

would be written:

 namespace xlink = "http://www.w3.org/1999/xlink"
 namespace dc = "http://purl.org/dc/elements/1.1/"

 start =
    [[
       xlink:type = "simple"
       xlink:role = "http://www.w3.org/1999/xhtml"
       xlink:arcrole = "http://www.rddl.org/purposes#reference"
       xlink:href = "library.xhtml"
       dc:title [[ "The library element" ]
       dc:author [[ "Eric van der Vlist" ]
    ]
    element library { book-element+ }

Again, note how the annotations are preceding the "element" pattern to indicate that they are the first children elements in the XML syntax. This applies also to annotations through foreign attributes of the grammar pattern, such as:

 <?xml version="1.0" encoding="utf-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0"
          xmlns:xlink="http://www.w3.org/1999/xlink"
          xlink:type="simple"
          xlink:role="http://www.w3.org/1999/xhtml"
          xlink:arcrole="http://www.rddl.org/purposes#reference"
          xlink:href="grammar.xhtml">
  .../...
 </grammar>

In this case, to be able to define the annotations before the grammar pattern, we need to make it explicit which is usually not necessary with the compact syntax:

 namespace xlink = "http://www.w3.org/1999/xlink"

 [[
   xlink:type = "simple"
   xlink:role = "http://www.w3.org/1999/xhtml"
   xlink:arcrole = "http://www.rddl.org/purposes#reference"
   xlink:href = "grammar.xhtml"
 ]
 grammar {
   .../...
 }

!Following annotations

So far so good, but then how do we define annotations which are not initial nor grammar annotations, such as in:

   <define name="author-element">
     <element name="author">
       <attribute name="id"/>
       <ref name="name-element"/>
       <ref name="born-element"/>
       <xhtml:p>After this point, everything is optional.</xhtml:p>
       <optional>
         <ref name="died-element"/>
       </optional>
     </element>
   </define>

This is done using a third syntax reserved to "following annotations" and here we would write:

 author-element =
   element author {
     attribute id { text },
     name-element,
     born-element >> xhtml:p [[ "After this point, everything is optional." ],
     died-element?
   }

Note the new syntax '>> xhtml:p [[ "After this point, all is optional." ]' with the leading ">>" being the indication that we have a "following annotation". As we see here, the following annotation is inserted where it is seen as a "following sibling" of the parent element representing the pattern in the XML syntax.

!All together

To wrap-up, let's have a look at the following schema snippet where annotations have been added pretty much in each location where there was room for them:

 <?xml version="1.0" encoding="utf-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0"
          xmlns:ann="http://dyomedea.com/examples/ns/annotations"
          ann:attribute="Annotation as foreign attribute for 'grammar'">
   <ann:element>Initial annotation as foreign element for "grammar"</ann:element>
   <start ann:attribute="Annotation as a foreign attribute for 'start'">
     <ann:element>Initial annotation as foreign element for "start"</ann:element>
     <element name="library" ann:attribute="Annotation as a foreign attribute for 'element'">
       <ann:element>Initial annotation as foreign element for "element"</ann:element>
       <oneOrMore ann:attribute="Annotation as a foreign attribute for 'oneOrMore'">
         <ann:element>Initial annotation as foreign element for "oneOrMore"</ann:element>
         <ref name="book-element" ann:attribute="Annotation as a foreign attribute for 'ref'">
           <ann:element>Initial annotation as foreign element for "ref"</ann:element>
         </ref>
         <ann:element>Following annotation as foreign element for "oneOrMore"</ann:element>
       </oneOrMore>
       <ann:element>Following annotation as foreign element for "element"</ann:element>
     </element>
     <ann:element>Following annotation as foreign element for "start"</ann:element>
   </start>
   <ann:element>Grammar annotation as foreign element for "grammar"</ann:element>
   .../...
 </grammar>

The compact syntax would be:

 namespace ann = "http://dyomedea.com/examples/ns/annotations"

 [[
   ann:attribute = 'Annotation as foreign attribute for "grammar"'
   ann:element [[ 'Initial annotation as foreign element for "grammar"' ]
 ]
 grammar {
   [[
     ann:attribute = "Annotation as a foreign attribute for 'start'"
     ann:element [[ 'Initial annotation as foreign element for "start"' ]
   ]
   start =
     [[
       ann:attribute = "Annotation as a foreign attribute for 'element'"
       ann:element [[
         'Initial annotation as foreign element for "element"'
       ]
     ]
     element library {
       [[
         ann:attribute =
           "Annotation as a foreign attribute for 'oneOrMore'"
         ann:element [[
           'Initial annotation as foreign element for "oneOrMore"'
         ]
       ]
       ([[
          ann:attribute = "Annotation as a foreign attribute for 'ref'"
          ann:element [[
            'Initial annotation as foreign element for "ref"'
          ]
        ]
        book-element
        >> ann:element [[
             'Following annotation as foreign element for "oneOrMore"'
           ]+)
       >> ann:element [[
            'Following annotation as foreign element for "element"'
          ]
     }
     >> ann:element [[
          'Following annotation as foreign element for "start"'
        ]
   ann:element [[ 'Grammar annotation as foreign element for "grammar"' ]
   .../...
  }

Impressive, isn't it? Although this syntax is strictly equivalent to the XML syntax, it's quite difficult to read and very tough to say where each of these annotations do belong. I think that it's a good enough demonstration that if application specific syntaxes may be defined which are more concise and easier to read than XML, when there is a need for extensibility and interoperability, XML is a clear winner.

!When initial annotations turn into following annotations

A last riddle before we move on... What do you think this annotation would mean?

      element born {
        xsd:date {
          [[
            xhtml:p [[
              "Add new parameters here to define a range."
            ]
          ]
          pattern = "[[0-9]{4}-[[0-9]{2}-[[0-9]{2}"
        }
      }

It can't be a following annotation on the "pattern" parameter since parameters have a text only content model and can't accept foreign elements and Relax NG considers that in this case, this is a following annotation and that it's equivalent to:

          <element name="born">
            <data type="date">
              <param name="pattern">[[0-9]{4}-[[0-9]{2}-[[0-9]{2}</param>
              <xhtml:p>Add new parameters here to define a range.</xhtml:p>
            </data>
          </element>

Note that this would also apply to the "value" pattern and that in both cases, the syntax using a following annotation cannot be used in the compact syntax.

!!Annotating Groups of Definitions

One may want to annotate a group of patterns. When these patterns are definitions of named patterns in a grammar and that compositors such as "group", "interleave" or "choice" cannot be used as a container for the annotation, Relax NG provides a "div" pattern for this purpose:

 <?xml version="1.0" encoding="UTF-8"?>
 <grammar xmlns:xhtml="http://www.w3.org/1999/xhtml" xmlns="http://relaxng.org/ns/structure/1.0">
 .../...
   <div>
     <xhtml:p>The content of the book element has been split in two named patterns:</xhtml:p>
     <define name="book-start">
       <attribute name="id"/>
       <ref name="isbn-element"/>
       <ref name="title-element"/>
       <zeroOrMore>
         <ref name="author-element"/>
       </zeroOrMore>
     </define>
     <define name="book-end">
       <zeroOrMore>
         <ref name="author-element"/>
       </zeroOrMore>
       <zeroOrMore>
         <ref name="character-element"/>
       </zeroOrMore>
       <attribute name="available"/>
     </define>
   </div>
 .../...
 </grammar>

or:

 [[
   xhtml:p [[
     "The content of the book element has been split in two named patterns:"
   ]
 ]
 div {
   book-start =
     attribute id { text },
     isbn-element,
     title-element,
     author-element*
   book-end =
     author-element*,
     character-element*,
     attribute available { text }
 }

The "div" pattern has no other effect than to group both definitions in a container so that annotations can be applied to the whole container instead of being applied to the individual definitions. Each of the embedded definition is still considered as global to the grammar and can be referenced as if they had not been wrapped into a "div" pattern.

!!Alternatives and Workarounds

All this seems pretty good, why would we like to find alternatives and workarounds? I can see two types of reasons for this: the first to take advantage of more generic mechanisms defined for XML and the other deals with the impossibility to annotate "value" and "param" patterns with foreign elements.

!Why reinvent XML 1.0 comments and PIs?

There is a tendency in recent XML applications to de facto deprecate the usage of XML comments and Processing Instructions (PIs) and to replace them by XML elements and attributes. There are often some good reasons to do so: using elements is more flexible when structured content needs to be added and the lack of support of namespaces for PIs makes it difficult to rely on their names which might be reused with different meanings by different applications. However, this doesn't mean that they shouldn't be used in Relax NG schemas.

Comments are fully supported and XML comments even have their equivalent in the compact syntax:

   <define name="author-element">
     <!-- Definition of the author element -->
     <element name="author">
       <attribute name="id"/>
       <ref name="name-element"/>
       <ref name="born-element"/>
       <optional>
         <ref name="died-element"/>
       </optional>
     </element>
   </define>

is equivalent to:

author-element =
  # Definition of the author element
  element author {
    attribute id { text },
    name-element,
    born-element,
    died-element?
  }
Note how, like in Unix shells, comments are marked by a hash ("#") in the compact syntax.

We could discuss endlessly whether this is better or worse than a counterpart based on foreign elements such as:

   <define name="author-element">
     <xhtml:p>Definition of the author element</xhtml:p>
     <element name="author">
       <attribute name="id"/>
       <ref name="name-element"/>
       <ref name="born-element"/>
       <optional>
         <ref name="died-element"/>
       </optional>
     </element>
   </define>

or:

 [[ xhtml:p [[ "Definition of the author element" ] ]
 author-element =
  element author {
    attribute id { text },
    name-element,
    born-element,
    died-element?
  }

I would tend to consider that the syntax for comments is much more readable in the compact syntax and that, even in the XML syntax, XML comments are more easy to be spotted as such with their syntax which is different from the XML elements. Readability is, of course, very subjective but there is no reason to refuse to use comments if you think that they are more readable. After all a simple XSLT transformation can transform comments into foreign elements and vice versa and getting good comments is more important than the syntax used to express them!

The issue would have been similar for PIs if they had an equivalent in the compact syntax. As comments, PIs may be considered as more readable than foreign elements. For instance, if we compare:

   <define name="author-element">
    <?sql query="select name, birthdate, deathdate from tbl_author"?>
    <element name="author">
      <attribute name="id"/>
      <ref name="name-element"/>
      <ref name="born-element"/>
      <optional>
        <ref name="died-element"/>
      </optional>
    </element>
  </define>

and:

  <define name="author-element" >
    <sql:select xmlns:sql="http://www.extensibility.com/saf/spec/safsample/sql-map.saf">
       select name, birthdate,deathdate from tbl_author
    </sql:select>
    <element name="author">
      <attribute name="id"/>
      <ref name="name-element"/>
      <ref name="born-element"/>
      <optional>
        <ref name="died-element"/>
      </optional>
    </element>
  </define>

There doesn't seem to be that much reasons to prefer the second syntax over the first one except for the lack of namespace support already mentioned and a greater extensibility of foreign elements.

Unfortunately, PIs do not translate into the compact syntax and are trashed during the conversion. If you want to keep the possibility to use indifferently both the XML and the compact syntax you will thus need to avoid using PIs.

!Annotation of the "value" and "param" patterns

What if we need to annotate "value" and "param" patterns which do not accept foreign elements? There isn't much we can do except using foreign attributes or XML comments or PIs as seen in the previous section or moving the annotations to another location.

Comments can be used freely in this context:

          <element name="born">
            <data type="date">
              <param name="minInclusive">1900-01-01</param>
              <param name="maxInclusive">2099-12-31</param>
              <param name="pattern">
                <!-- We don't want timezones in our dates. -->
                [[0-9]{4}-[[0-9]{2}-[[0-9]{2}
              </param>
            </data>
          </element>

Or:

      element born {
        xsd:date {
          minInclusive = "1900-01-01"
          maxInclusive = "2099-12-31"
          pattern =
          # We don't want timezones in our dates.
            "[[0-9]{4}-[[0-9]{2}-[[0-9]{2}\x{a}"
        }
      }


We can also transform the foreign elements into the same attributes, for instance:

          <element name="born">
            <data type="date">
              <param name="minInclusive">1900-01-01</param>
              <param name="maxInclusive">2099-12-31</param>
              <param name="pattern" xhtml:p="We don't want timezones in our dates.">[[0-9]{4}-[[0-9]{2}-[[0-9]{2}</param>
            </data>
          </element>

or:

      element born {
        xsd:date {
          minInclusive = "1900-01-01"
          maxInclusive = "2099-12-31"
          [[ xhtml:p = "We don't want timezones in our dates." ]
          pattern = "[[0-9]{4}-[[0-9]{2}-[[0-9]{2}"
        }
      }

Of course, there is no such thing as a "xhtml:p" attribute, but the meaning seems straightforward enough! The downside if both workarounds is that we cannot extend them if we have structured content and want, for instance, add a link in our comment. In this case, we will need to locate the comment in a foreign element at a different location, for instance:

          <element name="born">
            <data type="date">
              <xhtml:p>We don't want timezones in our dates
              (see <xhtml:a href="ref.xhtml#dates">dates ref</xhtml:a> for additional info.</xhtml:p>
              <param name="minInclusive">1900-01-01</param>
              <param name="maxInclusive">2099-12-31</param>
              <param name="pattern">[[0-9]{4}-[[0-9]{2}-[[0-9]{2}</param>
            </data>
          </element>

or:

      element born {
        [[
          xhtml:p [[
            "We don't want timezones in our dates (see "
            xhtml:a [[ href = "ref.xhtml#dates" "dates ref" ]
            " for additional info."
          ]
        ]
        xsd:date {
          minInclusive = "1900-01-01"
          maxInclusive = "2099-12-31"
          pattern = "[[0-9]{4}-[[0-9]{2}-[[0-9]{2}"
        }
      }

Note that we have lost the relation between the annotation and the exact location where it applies. One of the ways to get this information back is to add an identifier to the annotation and use a mechanism such as XLink to define a link between our "param" element and the annotation:

          <element name="born">
            <data type="date">
              <xhtml:p id="dates-notz">We don't want timezones in our dates
               (see <xhtml:a href="ref.xhtml#dates">dates ref</xhtml:a> for additional info.</xhtml:p>
              <param name="minInclusive">1900-01-01</param>
              <param name="maxInclusive">2099-12-31</param>
              <param name="pattern" xlink:type="simple"
               xlink:arcrole="http://www.rddl.org/purposes#reference"
               xlink:href="#dates-notz" >[[0-9]{4}-[[0-9]{2}-[[0-9]{2}</param>
            </data>
          </element>

or:

      element born {
        [[
          xhtml:p [[
            id = "dates-notz"
            "We don't want timezones in our dates (see "
            xhtml:a [[ href = "ref.xhtml#dates" "dates ref" ]
            " for additional info."
          ]
        ]
        xsd:date {
          minInclusive = "1900-01-01"
          maxInclusive = "2099-12-31"
          [[
            xlink:type = "simple"
            xlink:arcrole = "http://www.rddl.org/purposes#reference"
            xlink:href = "#dates-notz"
          ]
          pattern = "[[0-9]{4}-[[0-9]{2}-[[0-9]{2}"
        }
      }

Another option is to change the rule of the game and state that the annotation do not apply to its parent element, but, if there is one, to the preceding element. We will see in the next section that Relax NG's DTD compatibility specification uses this trick! Applied to our example, this would lead to writing:

      element born {
        xsd:date {
          minInclusive = "1900-01-01"
          maxInclusive = "2099-12-31"
          [[
            xhtml:p [[
              "We don't want timezones in our dates (see "
              xhtml:a [[ href = "ref.xhtml#dates" "dates ref" ]
              " for additional info."
            ]
          ]
          pattern = "[[0-9]{4}-[[0-9]{2}-[[0-9]{2}"
        }
      }


!!!Documentation

After this long introduction to annotations, we can start seeing more applications of these. The first obvious application of annotations is for documentation. The issue of documenting applications is a long running problem with three different schools:

* The documentation and the code can be separated but in this case there is nothing really specific to documenting Relax NG schemas and this is out of the scope of this book.
* The code can be embedded in the documentation as proposed by the proponents of "Literate Programming" and this approach will be presented in our next chapter "Chapter 14: Generating Relax NG schemas".
* The documentation can be embedded within the code and this approach applied to Relax NG leads to annotations as covered in this section.

We've seen the technical basis of how these annotations may be included in Relax NG schemas and generating a documentation from these annotations is mainly a matter of writing a XSLT transformation to extract and format them according to your needs. Instead of going over the details of such transformations which would be out of the scope of this book, we will see here some examples of such annotations using different existing XML namespaces.

!!Comments

Many examples have already been supplied in the previous section, including this one:

   <define name="author-element">
     <!-- Definition of the author element -->
     <element name="author">
       <attribute name="id"/>
       <ref name="name-element"/>
       <ref name="born-element"/>
       <optional>
         <ref name="died-element"/>
       </optional>
     </element>
   </define>

which is equivalent to:

author-element =
  # Definition of the author element
  element author {
    attribute id { text },
    name-element,
    born-element,
    died-element?
  }

As seen in the previous section, comments have the benefit to be readable both in the XML and in the compact syntax. They can be easily extracted, not only from the XML syntax using a XSLT transformation or also from the compact syntax using regular expressions and provide a lightweight way to document Relax NG schemas. They are also the least intrusive mechanism to annotate schemas and can be used at any location in a schema, including within the text only patterns "value" and "param".

Their readability in the compact syntax is so much better than annotations that, taking the risk to be called a devotee of the past, I would recommend them by default to document Relax NG schemas when there is no other special requirement.

Their downsides are well known:

*The XML recommendation states that their treatment by applications is optional and some tools just ignore them. This was the case of early parsers and editors but the situation has been improving since the early days of XML and most if not all the XML parsers and editors do now respect XML comments.
*They can only contain plain text and no XML structures. In the context of a Relax NG schema, this is often not an issue and when needed conventions can easily be added to define specific structures, either like it is done for JavaDoc where special "tags" are prefixed by "@" or for Wiki Wiki Webs where links are expressed as "[[ link title|http://...link.location]".

!!Relax NG DTD Compatibility Comments

We have already mentioned the Relax NG DTD Compatibility specification in our chapter about external datatype libraries "Chapter 8: Datatype Libraries" where we have studied the DTD datatypes. There is more than that in this specification which also includes a way to specify comments which would be included in a DTD equivalent to the Relax NG schema and also an annotation to define default values which we will see later on in this chapter.

The DTD compatibility comments have a special status in that a namespace has been defined for them by the Relax NG Technical Committee and that a shortcut has been defined to provide a concise form in the compact syntax. Being annotations in the XML syntax and comment like in the compact syntax, they are thus a kind of middle solution between XML comments and Relax NG annotations.

When using the XML syntax, DTD Compatibility Comments are foreign elements from the namespace "http://relaxng.org/ns/compatibility/annotations/1.0". Their content is text only and they may be annotated using foreign namespace attributes. An example of schema using this feature is:

 <?xml version="1.0" encoding="utf-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0" 
          xmlns:a="http://relaxng.org/ns/compatibility/annotations/1.0">
   <a:documentation>Relax NG flat schema for our library</a:documentation>
   <start>
     <element name="library">
       <oneOrMore>
         <ref name="book-element"/>
       </oneOrMore>
     </element>
   </start>
   <define name="author-element">
     <a:documentation>Definition of the author element</a:documentation>
     <element name="author">
       <attribute name="id"/>
       <ref name="name-element"/>
       <ref name="born-element"/>
       <optional>
         <ref name="died-element"/>
       </optional>
     </element>
   </define>
   .../...
 </grammar>

An equivalent schema using the compact syntax is:

 ## Relax NG flat schema for our library
 grammar{

 start = element library { book-element+ }

 ## Definition of the author element
 author-element =
  element author {
    attribute id { text },
    name-element,
    born-element,
    died-element?
  }
  .../...
 }

Note the syntax with the leading double sharps ("##") analogous to the "/**" comments used in JavaDoc and also the fact that even though they look like comments, these are annotations which have the same meaning and rules than initial annotations and must precede the pattern to which they apply. In fact, this syntax is only a shortcut to the corresponding annotations and this is equivalent to:

 namespace a = "http://relaxng.org/ns/compatibility/annotations/1.0"

 [[ a:documentation [[ "Relax NG flat schema for our library" ] ]

 grammar {
  start = element library { book-element+ }

  [[ a:documentation [[ "Definition of the author element" ] ]
  author-element =
   element author {
     attribute id { text },
     name-element,
     born-element,
     died-element?
   }
   ../..
 }

Also note that this shortcut has the same restrictions than initial annotations and that they must precede all the initial annotations. It is possible to mix them with other types of annotations and write for instance:

 namespace a = "http://relaxng.org/ns/compatibility/annotations/1.0"

 a:documentation [[ "Relax NG flat schema for our library" ]
 start = element library { book-element+ }

 ## Definition of the author element
 author-element =
   element author {
     attribute id { text },
     name-element,
     born-element,
     died-element?
   }
 .../...

Up to now, we have seen examples of these compatibility comments which were the first child element in their parent and this has been hiding an important feature of these comments: they are using the last trick we've mentioned in the previous section about workarounds for annotating "param" and "value" patterns and apply to the preceding sibling from the Relax NG namespace when there is one. This means that if we want to annotate the reference to the "name-element" definition, we can either write:

  <define name="author-element">
    <element name="author">
      <attribute name="id"/>
      <ref name="name-element">
        <a:documentation>Definition of the author element</a:documentation>
      </ref>
      <ref name="born-element"/>
      <optional>
        <ref name="died-element"/>
      </optional>
    </element>
  </define>

or:

  <define name="author-element">
    <element name="author">
      <attribute name="id"/>
      <ref name="name-element"/>
      <a:documentation>Definition of the author element</a:documentation>
      <ref name="born-element"/>
      <optional>
        <ref name="died-element"/>
      </optional>
    </element>
  </define>

In the first case, the DTD compatibility annotation is the first child element of its parent element ("ref") and applies to the "ref" pattern for this reason. In the second case, the annotation is not the first child element from the Relax NG namespace and applies to his preceding sibling, which is the "ref" pattern again.

The compact syntax has the same rules, and the following annotations are equivalent too:

author-element =
  element author {
    attribute id { text },

    ## Definition of the author element
    name-element,
    born-element,
    died-element?
  }

and:

author-element =
  element author {
    attribute id { text },
    name-element
    >> a:documentation [[ "Definition of the author element" ],
    born-element,
    died-element?
  }

Here again, a following annotation is considered as an annotation of the "name-element" reference.

Of course, if we were annotating a "param" or "value" pattern we would have no other choice than to locate the annotation after the pattern and this is why this tricky mechanism has been introduced.

!!XHTML Annotations

XHTML seems like a natural choice for embedding documentation in Relax NG schemas and we have already seen several examples of such annotations. The main benefit of XHTML is to be so similar to HTML that it is known by pretty much anyone who has ever published a web page. A lot of documentation and books are available on XHTML and many editors can be used to edit XHTML documents. Furthermore, if you keep to a reasonable subset of XHTML (such as for instance XHTML Basic), you have a simple and generic language to write documentation. And of course, the work needed if you want to publish the result of the extraction of XHTML annotations as XHTML is minimal since your annotations are already XHTML.

You will find more information about XHTML on the W3C web site: http://www.w3.org/MarkUp/ as well as in specialized books such as "HTML & XHTML: The Definitive Guide" (O'Reilly) and "XHTML: Moving Toward XML" (M&T Books).

We have also seen many examples of XHTML annotations, such as:


 <?xml version="1.0" encoding="utf-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0" xmlns:xhtml="http://www.w3.org/1999/xhtml">
   <xhtml:div>
     <xhtml:h1>Relax NG flat schema for our library</xhtml:h1>
     <xhtml:p>This schema has been written by <xhtml:a href="http://dyomedea.com/vdv">Eric van der Vlist</xhtml:a>.</xhtml:p>
   </xhtml:div>
   .../...
 </grammar>

or, using the compact syntax:

 namespace xhtml = "http://www.w3.org/1999/xhtml"

 xhtml:div
 [[
    xhtml:h1 [[ "Relax NG flat schema for our library" ]
    xhtml:p
    [[
       "This schema has been written by "
       xhtml:a [[ href = "http://dyomedea.com/vdv" "Eric van der Vlist" ]
       "."
    ]
 ]

 start = element library { book-element+ }
 .../...

Beyond the syntax which has already been discussed in the first part of this chapter, note how we have embedded a title ("xhtml:h1") and a paragraph ("xhtml:p") within a divisions ("xhtml:div"). This is generally a good practice which makes it easier to associate the title with the rest of the content and to manipulate the annotation as a whole.

!!DocBook Annotations

First design as a SGML application, DocBook is now also a XML language very popular for writing technical documentations. Beyond features which can be compared to those of XHTML, DocBook offers many predefined bells and whistles to facilitate indexes and cross references, to say that a text is a snippet of source code, identify acronyms and many other things. These features can be emulated in XHTML using the "class" attribute, but in DocBook they are built-in from the beginning and people agree with their meaning.

You will find more information about DocBook on its web site: http://www.oasis-open.org/committees/docbook/ and in the book "DocBook: The Definitive Guide" (O'Reilly).

DocBook is defined as a DTD which doesn't use any namespace and that's not an issue since Relax NG allows annotations through elements without namespace. To give you an idea of what DocBook looks like as well as an example showing how to "undeclare" a namespace in XML, the following example would match more or less what we had written in XHTML in the previous one:

 <?xml version="1.0" encoding="utf-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0">
   <sect1 xmlns="">
     <title>Relax NG flat schema for our library</title>
     <para>This schema has been written by <xref linkend="vdv"/>.</para>
   </sect1>
   <start>
     <element name="library">
       <oneOrMore>
         <ref name="book-element"/>
       </oneOrMore>
     </element>
   </start>
   .../...
 </grammar>

Or, with the compact syntax:

 sect1 [[
   title [[ "Relax NG flat schema for our library" ]
   para [[
     "This schema has been written by "
     xref [[ linkend = "vdv" ]
     "."
   ]
 ]
 start = element library { book-element+ }
 .../...

!!Dublin Core Annotations

While XHTML and DocBook are great to include content as documentation, Dublin Core is widely used over the web to include metadata about all type of resources and includes a set of elements with a description of their semantics which provide interoperable information including notions very relevant in a schema such as the name of its authors, their organization, the date, the copyright associated with the schema or the subjects which are relevant. Dublin Core is very complementary to DocBook and XHTML and Dublin Core is often use in XHTML documents where it finds a natural fit in the "meta" elements.

You will find more information about Dublin Core on their site: http://dublincore.org/ .

In a Relax NG schema, Dublin Core elements may be included wherever it makes sense: under the "grammar" pattern, they will qualify the whole grammar while under an "element" pattern, they would be useful to qualify the specific element.

A more complete example is:

 <?xml version="1.0" encoding="UTF-8"?>
 <grammar xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns="http://relaxng.org/ns/structure/1.0">
   <dc:title>The library element</dc:title>
   <dc:creator>Eric van der Vlist</dc:creator>
   <dc:subject>library, book, relax ng</dc:subject>
   <dc:description>This Relax NG schema has been written as an example to show how Dublin Core elements may be used.</dc:description>
   <dc:date>2003-01-30</dc:date>
   <dc:language>en</dc:language>
   <dc:rights>Copyright Eric van der Vlist, Dyomedea.
 			 During development, I give permission for non-commercial copying for
 			 educational and review purposes.
 			 After publication, all text will be released under the
 			 Free Software Foundation GFDL.</dc:rights>
  .../...
 </grammar>

or:

namespace dc = "http://purl.org/dc/elements/1.1/"

 dc:title [[ "The library element" ]
 dc:creator [[ "Eric van der Vlist" ]
 dc:subject [[ "library, book, relax ng" ]
 dc:description [[
   "This Relax NG schema has been written as an example to show how Dublin Core elements may be used."
 ]
 dc:date [[ "2003-01-30" ]
 dc:language [[ "en" ]
 dc:rights [[
   "Copyright Eric van der Vlist, Dyomedea. \x{a}" ~
   "			 During development, I give permission for non-commercial copying for \x{a}" ~
   "			 educational and review purposes. \x{a}" ~
   "			 After publication, all text will be released under the \x{a}" ~
   "			 Free Software Foundation GFDL."
 ]
 .../...


!!SVG Annotations

There is no reason to limit ourselves to text and metadata and graphics can be included, such as:

 <?xml version="1.0" encoding="utf-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0" xmlns:svg="http://www.w3.org/2000/svg">
   <start>
     <element name="library">
       <oneOrMore>
         <ref name="book-element"/>
       </oneOrMore>
     </element>
   </start>
   <define name="author-element">
     <element name="author">
       <svg:svg>
         <svg:title>A typical author</svg:title>
         <svg:ellipse style="stroke:#000000; fill:#e3e000; stroke-width:2pt;" id="head" cx="280" cy="250" rx="110" ry="130"/>
         <svg:ellipse style="stroke:none; fill:#7f7f7f; " id="leftEye" cx="240" cy="225" rx="18" ry="18"/>
         <svg:ellipse style="stroke:none; fill:#7f7f7f; " id="rightEye" cx="320" cy="225" rx="18" ry="18"/>
         <svg:path style="fill:none;stroke:#7F7F7F; stroke-width:5pt;" id="mouth" d="M 222 280 A 58 48 0 0 0 338 280"/>
       </svg:svg>
       <attribute name="id"/>
       <ref name="name-element"/>
       <ref name="born-element"/>
       <optional>
         <ref name="died-element"/>
       </optional>
     </element>
   </define>
   .../...
 </grammar>

Or, using the compact syntax:

 namespace svg = "http://www.w3.org/2000/svg"

 start = element library { book-element+ }
 author-element =
   [[
     svg:svg [[
       svg:title [[ "A typical author" ]
       svg:ellipse >[[
         style = "stroke:#000000; fill:#e3e000; stroke-width:2pt;"
         id = "head"
         cx = "280"
         cy = "250"
         rx = "110"
         ry = "130"
       ]
       svg:ellipse [[
         style = "stroke:none; fill:#7f7f7f; "
         id = "leftEye"
         cx = "240"
         cy = "225"
         rx = "18"
         ry = "18"
       ]
       svg:ellipse [[
         style = "stroke:none; fill:#7f7f7f; "
         id = "rightEye"
         cx = "320"
         cy = "225"
         rx = "18"
         ry = "18"
       ]
       svg:path [[
         style = "fill:none;stroke:#7F7F7F; stroke-width:5pt;"
         id = "mouth"
         d = "M 222 280 A 58 48 0 0 0 338 280"
       ]
     ]
   ]
   element author {
     attribute id { text },
     name-element,
     born-element,
     died-element?
   }
  .../...

I leave it to you as an additional exercise to visualize what a typical author looks like! Note we could have included a UML representation of the author element instead.

The Scalable Vector Graphics (SVG) is a XML vocabulary published by the W3C. You will find more information about SVG on its web site: http://www.w3.org/Graphics/SVG/ as well as in the book "SVG Essentials" (O'Reilly).

!!RDDL Annotations

The last type of annotation I'd like to mention here is a good transition between annotations for documentation purposes which we are seeing in this section and annotation for applications which we will see in the next section: RDDL has been designed as a XML vocabulary which can be used both by humans as documentation and by applications! Although RDDL has been invented to document namespaces, it can find a very good fit in a Relax NG schema from which it can be extracted to constitute RDDL documentations for the namespaces described in the schema. RDDL is based on XHTML and XLink and plays well with XHTML documentation.

You will find more information about RDDL on its web site: http://rddl.org .

RDDL main benefit is to provide a way to associate resources with a document and could be used to associate for instance a XSLT template and a CSS style to the definition of the author element:

 <?xml version="1.0" encoding="utf-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0"
          xmlns:xlink="http://www.w3.org/1999/xlink"
          xmlns:rddl="http://www.rddl.org/"
          xmlns:xhtml="http://www.w3.org/1999/xhtml">
   <start>
     <element name="library">
       <oneOrMore>
         <ref name="book-element"/>
       </oneOrMore>
     </element>
   </start>
   <define name="author-element">
     <element name="author">
       <xhtml:div>
         <rddl:resource id="author-transform" xlink:arcrole="http://www.w3.org/1999/xhtml"
           xlink:role="http://www.w3.org/1999/XSL/Transform" xlink:title="Author XSLT template"
           xlink:href="library.xslt#author">
           <xhtml:div class="resource">
             <xhtml:h4>XSLT Transformation</xhtml:h4>
             <xhtml:p>This <xhtml:a href="library.xslt#author">XSLT template</xhtml:a> displays the description of an author as XHTML.</xhtml:p>
           </xhtml:div>
         </rddl:resource>
         <rddl:resource id="CSS" xlink:title="CSS Stylesheet"
           xlink:role="http://www.isi.edu/in-notes/iana/assignments/media-types/text/css" xlink:href="author.css">
           <xhtml:div class="resource">
             <xhtml:h4>CSS Stylesheet</xhtml:h4>
             <xhtml:p>A <xhtml:a href="author.css">CSS stylesheet</xhtml:a> defining some cool styles to display an author.</xhtml:p>
           </xhtml:div>
         </rddl:resource>
       </xhtml:div>
       <attribute name="id"/>
       <ref name="name-element"/>
       <ref name="born-element"/>
       <optional>
         <ref name="died-element"/>
       </optional>
     </element>
   </define>
   .../...
 </grammar>

or:

 namespace rddl = "http://www.rddl.org/"
 namespace xhtml = "http://www.w3.org/1999/xhtml"
 namespace xlink = "http://www.w3.org/1999/xlink"

 start = element library { book-element+ }
 author-element =
   [[
     xhtml:div [[
       rddl:resource [[
         id = "author-transform"
         xlink:arcrole = "http://www.w3.org/1999/xhtml"
         xlink:role = "http://www.w3.org/1999/XSL/Transform"
         xlink:title = "Author XSLT template"
         xlink:href = "library.xslt#author"
         xhtml:div [[
           class = "resource"
           xhtml:h4 [[ "XSLT Transformation" ]
           xhtml:p [[
             "This "
             xhtml:a [[ href = "library.xslt#author" "XSLT template" ]
             " displays the description of an author as XHTML."
           ]
         ]
       ]
       rddl:resource [[
         id = "CSS"
         xlink:title = "CSS Stylesheet"
         xlink:role =
           "http://www.isi.edu/in-notes/iana/assignments/media-types/text/css"
         xlink:href = "author.css"
         xhtml:div [[
           class = "resource"
           xhtml:h4 [[ "CSS Stylesheet" ]
           xhtml:p [[
             xhtml:a [[ href = "author.css" "CSS stylesheet" ]
             " defining some cool styles to display an author."
           ]
         ]
       ]
     ]
   ]
   element author {
     attribute id { text },
     name-element,
     born-element,
     died-element?
   }
  .../...

!!!Annotation for applications

As mentioned in the introduction of this chapter, common uses of annotations by applications include using them as pre-processing instructions, as helpers for generating other schemas out of a Relax NG schema and as extensions to Relax NG itself.

!!Annotations for pre-processing

An interesting application of annotation for pre-processing has been proposed by Bob DuCharme and can be used to derive specific schemas by restriction out of a generic schema. The benefits of this approach are that it is extremely simple an provides a very straightforward workaround to the lack of derivation by restriction of Relax NG. It is also language neutral and can be applied to other schema languages such as W3C XML Schema where it is much simpler than the derivation by restriction feature built into the language.

You can find Bob DuCharme's proposal on the web: http://www.snee.com/xml/schemaStages.html and can download the XSLT transformation implementing it at http://www.snee.com/xml/schemaStages.zip.

The idea is to add annotations in elements which needs to be removed in a variant of the schema and to use these annotations to generate the different variants using a XSLT transformation. Each variant is called a stage. The list of the available stages is declared in a "sn:stages" element and for each element which is conditional, the list of the stages in which it needs to be kept is declared through a "sn:stages" attributes.

Since this technique is using annotations, the global schema can still be a valid schema which will validate a superset of the instance documents valid per each of the stages.

If we wanted to derive schemas requiring either a "book", "author", "library" or "character" element or both "book" or "author" as a document element from a generic schema allowing any of these, we could write:

 <?xml version="1.0" encoding="utf-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0" xmlns:sn="http://www.snee.com/ns/stages">
   <sn:stages>
     <sn:stage name="library"/>
     <sn:stage name="book"/>
     <sn:stage name="author"/>
     <sn:stage name="character"/>
     <sn:stage name="author-or-book"/>
   </sn:stages>
   <start>
     <choice>
       <ref name="library-element" sn:stages="library"/>
       <ref name="book-element" sn:stages="book author-or-book"/>
       <ref name="author-element" sn:stages="author author-or-book"/>
       <ref name="character-element" sn:stages="character"/>
     </choice>
   </start>
   .../...
 </grammar>

or:

 namespace sn = "http://www.snee.com/ns/stages"

 sn:stages [[
   sn:stage [[ name = "library" ]
   sn:stage [[ name = "book" ]
   sn:stage [[ name = "author" ]
   sn:stage [[ name = "character" ]
   sn:stage [[ name = "author-or-book" ]
 ]
 start =
   [[ sn:stages = "library" ] library-element
   | [[ sn:stages = "book author-or-book" ] book-element
   | [[ sn:stages = "author author-or-book" ] author-element
   | [[ sn:stages = "character" ] character-element
 .../...

This schema is a valid Relax NG schema which would accept any of these elements as a root. A transformation of the XML syntax through the XSLT transformation "getStage.xsl" provided in the zip file mentioned above with a parameter "stageName" set to "author-or-book" would remove all the elements with a "sn:stage" attribute that do not have "author-or-book" in their list of values:

 $ xsltproc --stringparam stageName author-or-book getStage.xsl doc-snee.rng
 <?xml version="1.0"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0" xmlns:sn="http://www.snee.com/ns/stages">

  <start>
    <choice>

      <ref name="book-element"/>
      <ref name="author-element"/>

    </choice>
  </start>
 .../...
 </grammar>

This transformation has thus performed a restriction on the schema and as many schemas can be generated this way as stages have been declared in the "sn:stages" element.

!!Annotations for conversion

We will see in "Appendix B: Using Relax NG As a Pivot Format" that Relax NG is a good fit to be a "pivot" format, i.e. a reference format in which schemas are kept and transformed into other languages. One of the limits of this approach is that features which are part of the target languages and not part of Relax NG seem to be out of reach and that would be true if we had no annotations. The two most notorious examples of such annotations are for generating DTDs and W3C XML Schema.

!Annotations to generate DTDs

This is the third and last facet of the DTD Compatibility specifications and this deals with default values for attributes which can be declared using a "a:defaultValue" attribute:

 <?xml version="1.0" encoding="utf-8"?>
 <element xmlns="http://relaxng.org/ns/structure/1.0" xmlns:a="http://relaxng.org/ns/compatibility/annotations/1.0" name="library">
   <oneOrMore>
     <element name="book">
       <attribute name="id"/>
        <optional>
         <attribute name="available" a:defaultValue="true">
           <choice>
             <value>true</value>
             <value>false</value>
           </choice>
         </attribute>
       </optional>
       .../...
     </element>
   </oneOrMore>
    .../...
 </element>

or:

 namespace a = "http://relaxng.org/ns/compatibility/annotations/1.0"

 element library {
   element book {
     attribute id { text },
     [[ a:defaultValue = "true" ]
     attribute available { "true" | "false" }?,
     element isbn { text },
     element title {
       attribute xml:lang { text },
       text
     },
     .../...
   }+
 }

The attribute needs to be declared as optional to use this feature and that means that there is no impact on the validation by a Relax NG processor. However, converters such as Trang will use this annotation to generate a default value in a DTD:

 <!ATTLIST book
  id CDATA #REQUIRED
  available (true|false) 'true'>

!Annotations to generate W3C XML Schema schemas

There is no official specification about how to generate W3C XML Schema schemas from Relax NG and what we will say in this small section is derived from the documentation of Trang available on the web at http://www.thaiopensource.com/relaxng/trang-manual.html.

The first thing to note is that Trang does support the "a:defaultValue" attribute and that the schema presented above would be translated as:

  <xs:element name="book">
    <xs:complexType>
      <xs:sequence>
        <xs:element ref="isbn"/>
        <xs:element ref="title"/>
        <xs:element minOccurs="0" maxOccurs="unbounded" ref="author"/>
        <xs:element minOccurs="0" maxOccurs="unbounded" ref="character"/>
      </xs:sequence>
      <xs:attribute name="id" use="required"/>
      <xs:attribute name="available" default="true">
        <xs:simpleType>
          <xs:restriction base="xs:token">
            <xs:enumeration value="true"/>
            <xs:enumeration value="false"/>
          </xs:restriction>
        </xs:simpleType>
      </xs:attribute>
    </xs:complexType>
  </xs:element>

Note the "default" attribute in the declaration of the "available" attribute.

In addition to this annotation, James Clark has created a specific namespace: http://www.thaiopensource.com/ns/relaxng/xsd to control the translation to W3C XML Schema. This translation is far from being obvious and a Relax NG schema can often be translated using different features of W3C XML Schema. James Clark has made a lot of choices in his implementation based on best practices, but there are still some options which are really context dependent and for which the users may be given a choice.

In the current version (as of 30 January 2003), there is only one annotation attribute available to perform such choices, the "tx:enableAbstractElements" attribute which may be included in "grammar", "div" or "include". This attribute can take the values "true" or "false" and controls whether abstract elements may be used in substitution groups. This is a fairly advanced feature of W3C XML Schema and we won't present it here nor give any example. You will find more information on this feature in my tutorial on XML.com: http://xml.com/pub/a/2000/11/29/schemas/part1.html or in my book "XML Schema: The W3C Object-Oriented Descriptions for XML" (O'Reilly).

The Trang manual indicates that more annotations might be added in the future.

!Schema Adjunct Framework

The Schema Adjunct Framework (SAF) is more or less falling into this category as well. SAF is a generic framework to store processing information in relation with schemas and can work either as standalone or as "schema adornments", i.e. annotations embedded in schemas. Although it has been developed to work with W3C XML Schema, there is no reason that it couldn't be used to adorn Relax NG schemas.

You can find more information about SAF on the web: http://www.tibco.com/solutions/products/extensibility/resources/saf.jsp

The momentum behind SAF seems to have decreased a lot since end of 2001, but this is definitely something to check if you need to add processing information in a schema. A simplified example for a SAF adornment in Relax NG could be:

  <define name="author-element">
    <sql:select>select <sql:elem>name</sql:elem>, <sql:elem>birthdate</sql:elem>, <sql:elem>deathdate</sql:elem>
      from tbl_author</sql:select>
    <element name="author">
      <attribute name="id"/>
      <ref name="name-element"/>
      <ref name="born-element"/>
      <optional>
        <ref name="died-element"/>
      </optional>
    </element>
  </define>

or:

 [[
   sql:select [[
     "select "
     sql:elem [[ "name" ]
     ", "
     sql:elem [[ "birthdate" ]
     ", "
     sql:elem [[ "deathdate" ]
     " from tbl_author"
   ]
 ]
 author-element =
   element author {
     attribute id { text },
     name-element,
     born-element,
     died-element?
   }


!!Annotations for extension

Annotations can also be used as extensions to influence the behavior of the Relax NG processors which support them which is more controversial but can also be very useful. The two applications which I am aware of in this category are for embedding Schematron rule and my own XVIF project which allows to define pipes of validations and transformations that act as Relax NG patterns.

!Embedded Schematron rules

Schematron is a XML schema language rather untypical since instead of being grammar based like Relax NG and focus on describing documents, Schematron is rule based and consists in lists of rules to check on documents. Giving the exhaustive list of all the rules needed to validate a document is a very verbose and error prone task but on the other hand, the ability to write your own rules gives a flexibility and a power which can't be matched by a grammar based schema language.  The two types of languages appear thus to be more complementary than competitors and using both together allows to take the best of each of them.

You will find more information about Schematron on its web site: http://www.ascc.net/xml/resource/schematron/schematron.html.

Schematron is a good fit, for instance, if we want to check that the "id" attribute of our book element is composed of the ISBN number prefixed by the letter "b". In this case we would write:

 <?xml version="1.0" encoding="utf-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0" xmlns:s="http://www.ascc.net/xml/schematron">
   <define name="book-element">
     <element name="book">
       <s:rule context="book">
         <s:assert test="@id = concat('b', isbn)"> The id needs to be the isbn number prefixed by &quot;b&quot; </s:assert>
       </s:rule>
       <attribute name="id"/>
       <attribute name="available"/>
       <ref name="isbn-element"/>
       <ref name="title-element"/>
       <zeroOrMore>
         <ref name="author-element"/>
       </zeroOrMore>
       <zeroOrMore>
         <ref name="character-element"/>
       </zeroOrMore>
     </element>
   </define>
   .../...
 </grammar>

or:

 namespace s = "http://www.ascc.net/xml/schematron"

 book-element =
   [[
     s:rule [[
       context = "book"
       s:assert [[
         test = "@id = concat('b', isbn)"
         ' The id needs to be the isbn number prefixed by "b" '
       ]
     ]
   ]
   element book {
     attribute id { text },
     attribute available { text },
     isbn-element,
     title-element,
     author-element*,
     character-element*
   }
 .../...

The Schematron annotation is composed of a "rule" element which is setting the context and embedded "assert" elements defining assertions. Instead of "assert", "report" elements can also be used which are the opposite of assertions and report errors when they are true. These checks are applied to all the elements meeting the XPath expression expressed in the "context" attribute of the "rule" elements and the "test" attribute of the "assert" or "report" elements are also XPath expressions.

At this point, we must note that there is a difference of appreciation between implementations on the scope in which the rules must be applied leading to potential issues of interoperability between implementations.

On one side, the Schematron specification states that when Schematron rules are embedded in another language, they must be collected and bundled into a Schematron schema independently of where they have been found in the original schema. In other words, this means that the rule which we has been defined above should be applied to all the "book" elements in the instance documents. This is the approach taken by the Topologi multi validator (see http://www.topologi.com/products/validator/index.html).

On the other side, when a schematron rule is embedded in a Relax NG "element" pattern like it is the case here, it is rather tempting to evaluate the rule in the context of the pattern. In that case, the rule will only apply to the "book" elements which are included in the context node and if the rule fails, the element pattern will fail and other alternatives will be checked. This is the approch taken by Sun's Multi Schema Validator (see http://wwws.sun.com/software/xml/developers/multischema/).

The difference can be seen in an example such as:

  <define name="book-element">
    <choice>
      <element name="book">
        <s:rule context="book">
          <s:assert test="@id = concat('b', isbn)"> The id needs to be the isbn number prefixed by "b" </s:assert>
        </s:rule>
        <attribute name="id"/>
        <attribute name="available"/>
        <ref name="isbn-element"/>
        <ref name="title-element"/>
        <zeroOrMore>
          <ref name="author-element"/>
        </zeroOrMore>
        <zeroOrMore>
          <ref name="character-element"/>
        </zeroOrMore>
      </element>
      <element name="book">
        <attribute name="id">
          <value>ggjh0836217462</value>
        </attribute>
        <attribute name="available"/>
        <ref name="isbn-element"/>
        <ref name="title-element"/>
        <zeroOrMore>
          <ref name="author-element"/>
        </zeroOrMore>
        <zeroOrMore>
          <ref name="character-element"/>
        </zeroOrMore>
      </element>
    </choice>
  </define>

In this case, the approach taken by the Schematron specification would lead to consider an instance document with a book id equal to "ggjh0836217462" as invalid since the evaluation of the Schematron rules is completely decoupled from the validation by the Relax NG schema and the approach taken by MSV would consider the same document as valid since it's meeting one of the alternative definitions for the "book" element.

!XVIF

The interoperability issue mentioned above is a good illustration of the difficulty to mix elements from different languages which have been specified independently and the XML Validation Interoperability Framework (XVIF) is a proposal for a framework which would take care of this kind of issues. You will find more information on XVIF at its home page: http://downloads.xmlschemata.org/python/xvif/.

The principle of XVIF is to define "micro pipes" of transformations and validations which can be embeded in different transformation and validation languages. When the host language is Relax NG, these micro pipes behave as Relax NG patterns.

There are many use cases for such micro pipes and one of them is to include transformations to fit text nodes into existing datatypes.  For instance, we have been lucky enough to have dates which are using the ISO 8601 format in our documents, but we could as well have had let's say French date formats. In this case, a set of regular expressions can be defined to do the transformation between these dates and the ISO 8601 format and XVIF gives a way to integrate these regular expressions in a Relax NG schema:

 <?xml version="1.0" encoding="utf-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0" xmlns:if="http://namespaces.xmlschemata.org/xvif/iframe"
          datatypeLibrary="http://www.w3.org/2001/XMLSchema-datatypes">
   <define name="born-element">
     <element name="born">
       <if:pipe>
         <if:validate type="http://namespaces.xmlschemata.org/xvif/regexp" 
                      apply="m/[[0-9]+ .+ [[0-9]+/"/>
         <if:transform type="http://namespaces.xmlschemata.org/xvif/regexp" 
                       apply="s/^[[ \t\n]*([[0-9] .*)$/0\1/"/>
         <if:transform type="http://namespaces.xmlschemata.org/xvif/regexp" 
                       apply="s/([[0-9]+) janvier ([[0-9]+)/\2-01-\1/"/>
         <if:transform type="http://namespaces.xmlschemata.org/xvif/regexp" 
                       apply="s/([[0-9]+) fevrier ([[0-9]+)/\2-02-\1/"/>
         <if:transform type="http://namespaces.xmlschemata.org/xvif/regexp" 
                       apply="s/([[0-9]+) mars ([[0-9]+)/\2-03-\1/"/>
         <if:transform type="http://namespaces.xmlschemata.org/xvif/regexp" 
                       apply="s/([[0-9]+) avril ([[0-9]+)/\2-04-\1/"/>
         <if:transform type="http://namespaces.xmlschemata.org/xvif/regexp" 
                       apply="s/([[0-9]+) mai ([[0-9]+)/\2-05-\1/"/>
         <if:transform type="http://namespaces.xmlschemata.org/xvif/regexp" 
                       apply="s/([[0-9]+) juin ([[0-9]+)/\2-06-\1/"/>
         <if:transform type="http://namespaces.xmlschemata.org/xvif/regexp" 
                       apply="s/([[0-9]+) juillet ([[0-9]+)/\2-07-\1/"/>
         <if:transform type="http://namespaces.xmlschemata.org/xvif/regexp" 
                       apply="s/([[0-9]+) aout ([[0-9]+)/\2-08-\1/"/>
         <if:transform type="http://namespaces.xmlschemata.org/xvif/regexp" 
                       apply="s/([[0-9]+) septembre ([[0-9]+)/\2-09-\1/"/>
         <if:transform type="http://namespaces.xmlschemata.org/xvif/regexp" 
                       apply="s/([[0-9]+) octobre ([[0-9]+)/\2-10-\1/"/>
         <if:transform type="http://namespaces.xmlschemata.org/xvif/regexp" 
                       apply="s/([[0-9]+) novembre ([[0-9]+)/\2-11-\1/"/>
         <if:transform type="http://namespaces.xmlschemata.org/xvif/regexp" 
                       apply="s/([[0-9]+) decembre ([[0-9]+)/\2-12-\1/"/>
         <if:validate type="http://relaxng.org/ns/structure/1.0">
           <if:apply>
             <data type="date">
               <param name="minInclusive">1900-01-01</param>
               <param name="maxInclusive">2099-12-31</param>
             </data>
           </if:apply>
         </if:validate>
       </if:pipe>
       <text if:ignore="1"/>
     </element>
   </define>
   .../...
 </grammar>

or:

 namespace if = "http://namespaces.xmlschemata.org/xvif/iframe"
 namespace rng = "http://relaxng.org/ns/structure/1.0"

 datatypes d = "http://relaxng.org/ns/compatibility/datatypes/1.0"

 born-element =
   [[
     if:pipe [[
       if:validate [[
         type = "http://namespaces.xmlschemata.org/xvif/regexp"
         apply = "m/[[0-9]+ .+ [[0-9]+/"
       ]
       if:transform [[
         type = "http://namespaces.xmlschemata.org/xvif/regexp"
         apply = "s/^[[ \t\n]*([[0-9] .*)$/0\1/"
       ]
       if:transform [[
         type = "http://namespaces.xmlschemata.org/xvif/regexp"
         apply = "s/([[0-9]+) janvier ([[0-9]+)/\2-01-\1/"
       ]
       if:transform [[
         type = "http://namespaces.xmlschemata.org/xvif/regexp"
         apply = "s/([[0-9]+) fevrier ([[0-9]+)/\2-02-\1/"
       ]
       if:transform [[
         type = "http://namespaces.xmlschemata.org/xvif/regexp"
         apply = "s/([[0-9]+) mars ([[0-9]+)/\2-03-\1/"
       ]
       if:transform [[
         type = "http://namespaces.xmlschemata.org/xvif/regexp"
         apply = "s/([[0-9]+) avril ([[0-9]+)/\2-04-\1/"
       ]
       if:transform [[
         type = "http://namespaces.xmlschemata.org/xvif/regexp"
         apply = "s/([[0-9]+) mai ([[0-9]+)/\2-05-\1/"
       ]
       if:transform [[
         type = "http://namespaces.xmlschemata.org/xvif/regexp"
         apply = "s/([[0-9]+) juin ([[0-9]+)/\2-06-\1/"
       ]
       if:transform [[
         type = "http://namespaces.xmlschemata.org/xvif/regexp"
         apply = "s/([[0-9]+) juillet ([[0-9]+)/\2-07-\1/"
       ]
       if:transform [[
         type = "http://namespaces.xmlschemata.org/xvif/regexp"
         apply = "s/([[0-9]+) aout ([[0-9]+)/\2-08-\1/"
       ]
       if:transform [[
         type = "http://namespaces.xmlschemata.org/xvif/regexp"
         apply = "s/([[0-9]+) septembre ([[0-9]+)/\2-09-\1/"
       ]
       if:transform [[
         type = "http://namespaces.xmlschemata.org/xvif/regexp"
         apply = "s/([[0-9]+) octobre ([[0-9]+)/\2-10-\1/"
       ]
       if:transform [[
         type = "http://namespaces.xmlschemata.org/xvif/regexp"
         apply = "s/([[0-9]+) novembre ([[0-9]+)/\2-11-\1/"
       ]
       if:transform [[
         type = "http://namespaces.xmlschemata.org/xvif/regexp"
         apply = "s/([[0-9]+) decembre ([[0-9]+)/\2-12-\1/"
       ]
       if:validate [[
         type = "http://relaxng.org/ns/structure/1.0"
         if:apply [[
           rng:data [[
             type = "date"
             rng:param [[ name = "minInclusive" "1900-01-01" ]
             rng:param [[ name = "maxInclusive" "2099-12-31" ]
           ]
         ]
       ]
     ]
   ]
   element born { [[ if:ignore = "1" ] text }

In this example, we have defined a pipe ("if:pipe") of twelve transformations ("if:transform") using regular expressions and each of them converting one of the twelve months and a final validation ("if:validate") which itself is using Relax NG to check that the result is a ISO 8601 date between 1900 and 2099. The "text" pattern has a "if:ignore" attribute showing to XVIF compliant processors that it's a fallback pattern for the other Relax NG processors.
