In the previous chapter (Chapter 12: Annotating Schemas) we have seen how information could be added to Relax NG schemas to make them more readable and also to improve the ability to extract information from the schemas and transform them into other useful documents such as documentation, diagrams or even applications. The underlying assumption up to now in this book has been that Relax NG was a natural level at which we would work and that we would be editing these schemas and this is certainly a valid point of view. However, a Relax NG schema (and any XML schema in general) is a physical model of a class of instance documents and we could also want to work at another level and generate our Relax NG schemas from this other level.

From this other point of view, Relax NG shines as an ideal choice for a target language because of its almost complete lack of restrictions. This lack of restrictions means that during the transformation of a model into a Relax NG schema, you won't have to worry with things such as "I must declare all my attributes after my elements", "I should take care to disallow unordered models in such and such circumstances", "if I have already declared this content here, I can't declare it again here", ... In other words, it will make your life much easier and let you concentrate on which document you want to get instead of having to worry about the constraints of the schema language.

What are these other levels on which we might want to work? In fact we could want to be either more concrete or more abstract than Relax NG and could either have a "bottom up" or "top down" approach. Proponents of a "bottom up" approach would enjoy working with instance documents rather than with schemas and Examplotron has been designed for them while the adepts of a "top down" approach will want to work at a higher level and use a methodology such as UML to model their documents. These two approaches may lead to many other variants and we will also see how literate programmers can include Relax NG patterns in their documentations and when you may want to replace your Relax NG schema by a simple spreadsheet.

!!!Examplotron: the instance document is its own schema

I have created Examplotron in March 2001 from a very simple idea: when you want to describe the element "foo", why would you have to learn yet another language and write "<element name='foo'><empty/></element>' or 'element foo {empty}'? Wouldn't it be so much simpler to just write the element in plain XML: "<foo/>"? Or, in other terms, instead of describing instance documents, why couldn't we show them?

The first implementation, published with the original description of Examplotron relied on a double XSLT transformation: the Examplotron "schema" was compiled by a XSLT transformation into another XSLT transformation which performed the validation of the instance documents. The concept has received many good comments when I have announced it, but nothing very significant did happen: moving along to add new features would have meant to redefine the full semantics of a new schema language and the implementation as a XSLT transformation was becoming very complex and the project was stalled until I realized the potential of using Relax NG as a target format.

Since its release 0.5, Examplotron is now implemented as a XSLT transformation transforming its schema into Relax NG and due to the potential of this approach, Examplotron has made more progress in two weeks than in two years under the previous architecture!

!!Ten minutes guide to Examplotron

Let's take a snippet of our example document:

 <?xml version="1.0" encoding="utf-8"?>
 <character id="Snoopy">
   <name>Snoopy</name>
   <born>1950-10-04</born>
   <qualification>extroverted beagle</qualification>
 </character>

The good news is that this is an Examplotron schema and to get an idea of what this schema means, we can translate it into a Relax NG schema:

 <?xml version="1.0" encoding="UTF-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0"
  xmlns:ega="http://examplotron.org/annotations/"
  xmlns:sch="http://www.ascc.net/xml/schematron"
  datatypeLibrary="http://www.w3.org/2001/XMLSchema-datatypes">
    <start>
       <element name="character">
          <optional>
             <attribute name="id">
                <ega:example id="Snoopy"/>
             </attribute>
          </optional>
          <element name="name">
             <text>
                <ega:example>Snoopy</ega:example>
             </text>
          </element>
          <element name="born">
             <data type="date">
                <ega:example>1950-10-04</ega:example>
             </data>
          </element>
          <element name="qualification">
             <text>
                <ega:example>extroverted beagle</ega:example>
             </text>
          </element>
       </element>
    </start>
 </grammar>

or:

 namespace ega = "http://examplotron.org/annotations/"
 namespace sch = "http://www.ascc.net/xml/schematron"

 start =
   element character {
     [[ ega:example [[ id = "Snoopy" ] ] attribute id { text }?,
     element name { [[ ega:example [[ "Snoopy" ] ] text },
     element born { [[ ega:example [[ "1950-10-04" ] ] xsd:date },
     element qualification {
       [[ ega:example [[ "extroverted beagle" ] ] text
     }
   }

We see there that the Examplotron, schema has not only the same modeling power than its Relax NG counterpart, but that annotations need to be added to the Relax NG schema if we don't want to loose the "examples" provided in Examplotron which are useful for documentation purposes and also to allow the reverse transformation (Relax NG to Examplotron) if that was needed.

The other thing to note in this example is that Examplotron is making inferences from what is found in the schema: here, Examplotron has assumed that the order between "name", "born" and "qualification" is significant, that these elements are mandatory, that the id attribute is optional, that the "born" element has a type "xsd:date" and that all the other elements and attribute are just text. These assumptions are designed to catch what is most likely to have been the intention of the designer of the document. The idea behind this is to bet that most of the time, people won't have to do anything to tweak their Examplotron schema.

What's happening when Examplotron got it wrong? There is no magic here: if you want to go against the inferences of Examplotron, you need to say it. And the way to say it is through annotating the Examplotron schema. To say that the "qualification" element is optional, you would for instance add an "eg:occurs" attribute with a value "?" and to say that the "id" attribute has a type "dtd:ID", you would have to set its content to "{dtd:id}":

 <?xml version="1.0" encoding="utf-8"?>
 <character id="{dtd:id}" xmlns:eg="http://examplotron.org/0/">
   <name>Snoopy</name>
   <born>1950-10-04</born>
   <qualification eg:occurs="?">extroverted beagle</qualification>
 </character>

And this would be translated as:

 <?xml version="1.0" encoding="UTF-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0"
  xmlns:ega="http://examplotron.org/annotations/"
  xmlns:sch="http://www.ascc.net/xml/schematron"
  datatypeLibrary="http://www.w3.org/2001/XMLSchema-datatypes">
    <start>
       <element name="character">
          <optional>
             <attribute name="id">
                <data type="id" datatypeLibrary="http://relaxng.org/ns/compatibility/datatypes/1.0"/>
             </attribute>
          </optional>
          <element name="name">
             <text>
                <ega:example>Snoopy</ega:example>
             </text>
          </element>
          <element name="born">
             <data type="date">
                <ega:example>1950-10-04</ega:example>
             </data>
          </element>
          <optional>
             <element name="qualification">
                <text>
                   <ega:example>extroverted beagle</ega:example>
                </text>
             </element>
          </optional>
       </element>
    </start>
 </grammar>

or:

 namespace ega = "http://examplotron.org/annotations/"
 namespace sch = "http://www.ascc.net/xml/schematron"

 datatypes d = "http://relaxng.org/ns/compatibility/datatypes/1.0"

 start =
   element character {
     attribute id { d:id }?,
     element name { [[ ega:example [[ "Snoopy" ] ] text },
     element born { [[ ega:example [[ "1950-10-04" ] ] xsd:date },
     element qualification {
       [[ ega:example [[ "extroverted beagle" ] ] text
     }?
   }

If you compare the compact syntax and the Examplotron schema, you see that we have something of similar conciseness, the compact syntax looking more formal and Examplotron looking more "visual". However, given the rules described in the documentation of Examplotron, these two schemas are equivalent and a round trip is possible (Examplotron to Relax NG and back from Relax NG to Examplotron).

One can go pretty far with these annotations as shown in this more complete example which defines contents as "interleave", mandatory attributes and defines the content of the complex elements as named patterns:

 <?xml version="1.0" encoding="utf-8"?>
 <library xmlns:eg="http://examplotron.org/0/" eg:content="eg:interleave" eg:define="library-content">
   <book available="true" eg:occurs="*" eg:define="book-content">
     <eg:attribute name="id" eg:content="dtd:id">b0836217462</eg:attribute>
     <isbn>0836217462</isbn>
     <title xml:lang="en">Being a Dog Is a Full-Time Job</title>
     <author eg:occurs="+" eg:define="author-content" eg:content="eg:interleave">
       <eg:attribute name="id" eg:content="dtd:id">CMS</eg:attribute>
       <name>Charles M Schulz</name>
       <born>1922-11-26</born>
       <died>2000-02-12</died>
     </author>
     <character eg:define="character-content" eg:content="eg:interleave">
       <eg:attribute name="id" eg:content="dtd:id">PP</eg:attribute>
       <name>Peppermint Patty</name>
       <born>1966-08-22</born>
       <qualification>bold, brash and tomboyish</qualification>
     </character>
     <character id="Snoopy">
       <name>Snoopy</name>
       <born>1950-10-04</born>
       <qualification>extroverted beagle</qualification>
     </character>
     <character id="Schroeder">
       <name>Schroeder</name>
       <born>1951-05-30</born>
       <qualification>brought classical music to the Peanuts strip</qualification>
     </character>
     <character id="Lucy">
       <name>Lucy</name>
       <born>1952-03-03</born>
       <qualification>bossy, crabby and selfish</qualification>
     </character>
   </book>
 </library>

The schema generated by this Examplotron is:

 <?xml version="1.0" encoding="UTF-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0" xmlns:ega="http://examplotron.org/annotations/" xmlns:sch="http://www.ascc.net/xml/schematron" datatypeLibrary="http://www.w3.org/2001/XMLSchema-datatypes">
    <start>
       <element name="library">
          <ref name="library-content" ega:def="true"/>
       </element>
    </start>
    <define name="library-content">
       <interleave>
          <zeroOrMore>
             <element name="book">
                <ref name="book-content" ega:def="true"/>
             </element>
          </zeroOrMore>
       </interleave>
    </define>
    <define name="book-content">
       <optional>
          <attribute name="available">
             <data type="boolean">
                <ega:example available="true"/>
             </data>
          </attribute>
       </optional>
       <attribute name="id">
          <ega:skipped>b0836217462</ega:skipped>
          <data type="id" datatypeLibrary="http://relaxng.org/ns/compatibility/datatypes/1.0"/>
       </attribute>
       <element name="isbn">
          <data type="integer">
             <ega:example>0836217462</ega:example>
          </data>
       </element>
       <element name="title">
          <optional>
             <attribute name="lang" ns="http://www.w3.org/XML/1998/namespace">
                <ega:example xml:lang="en"/>
             </attribute>
          </optional>
          <text>
             <ega:example>Being a Dog Is a Full-Time Job</ega:example>
          </text>
       </element>
       <oneOrMore>
          <element name="author">
             <ref name="author-content" ega:def="true"/>
          </element>
       </oneOrMore>
       <oneOrMore>
          <element name="character">
             <ref name="character-content" ega:def="true"/>
          </element>
       </oneOrMore>
       <ega:skipped>
          <character xmlns="" xmlns:eg="http://examplotron.org/0/" id="Snoopy">
             <name>Snoopy</name>
             <born>1950-10-04</born>
             <qualification>extroverted beagle</qualification>
          </character>
       </ega:skipped>
       <ega:skipped>
          <character xmlns="" xmlns:eg="http://examplotron.org/0/" id="Schroeder">
             <name>Schroeder</name>
             <born>1951-05-30</born>
             <qualification>brought classical music to the Peanuts strip</qualification>
          </character>
       </ega:skipped>
       <ega:skipped>
          <character xmlns="" xmlns:eg="http://examplotron.org/0/" id="Lucy">
             <name>Lucy</name>
             <born>1952-03-03</born>
             <qualification>bossy, crabby and selfish</qualification>
          </character>
       </ega:skipped>
    </define>
    <define name="author-content">
       <interleave>
          <attribute name="id">
             <ega:skipped>CMS</ega:skipped>
             <data type="id" datatypeLibrary="http://relaxng.org/ns/compatibility/datatypes/1.0"/>
          </attribute>
          <element name="name">
             <text>
                <ega:example>Charles M Schulz</ega:example>
             </text>
          </element>
          <element name="born">
             <data type="date">
                <ega:example>1922-11-26</ega:example>
             </data>
          </element>
          <element name="died">
             <data type="date">
                <ega:example>2000-02-12</ega:example>
             </data>
          </element>
       </interleave>
    </define>
    <define name="character-content">
       <interleave>
          <attribute name="id">
             <ega:skipped>PP</ega:skipped>
             <data type="id" datatypeLibrary="http://relaxng.org/ns/compatibility/datatypes/1.0"/>
          </attribute>
          <element name="name">
             <text>
                <ega:example>Peppermint Patty</ega:example>
             </text>
          </element>
          <element name="born">
             <data type="date">
                <ega:example>1966-08-22</ega:example>
             </data>
          </element>
          <element name="qualification">
             <text>
                <ega:example>bold, brash and tomboyish</ega:example>
             </text>
          </element>
       </interleave>
    </define>
 </grammar>

Or (skipping some annotation for readability):

 namespace eg = "http://examplotron.org/0/"
 namespace ega = "http://examplotron.org/annotations/"
 namespace sch = "http://www.ascc.net/xml/schematron"

 datatypes d = "http://relaxng.org/ns/compatibility/datatypes/1.0"

 start = element library { [[ ega:def = "true" ] library-content }
 library-content = element book { [[ ega:def = "true" ] book-content }*
 book-content =
   attribute available {
     [[ ega:example [[ available = "true" ] ] xsd:boolean
   }?,
   [[ ega:skipped [[ "b0836217462" ] ] attribute id { d:id },
   element isbn { [[ ega:example [[ "0836217462" ] ] xsd:integer },
   element title {
     [[ ega:example [[ xml:lang = "en" ] ] attribute lang { text }?,
     [[ ega:example [[ "Being a Dog Is a Full-Time Job" ] ] text
   },
   element author { [[ ega:def = "true" ] author-content }+,
   (element character { [[ ega:def = "true" ] character-content }+)
 author-content =
   [[ ega:skipped [[ "CMS" ] ] attribute id { d:id }
   & element name { [[ ega:example [[ "Charles M Schulz" ] ] text }
   & element born { [[ ega:example [[ "1922-11-26" ] ] xsd:date }
   & element died { [[ ega:example [[ "2000-02-12" ] ] xsd:date }
 character-content =
   [[ ega:skipped [[ "PP" ] ] attribute id { d:id }
   & element name { [[ ega:example [[ "Peppermint Patty" ] ] text }
   & element born { [[ ega:example [[ "1966-08-22" ] ] xsd:date }
   & element qualification {
       [[ ega:example [[ "bold, brash and tomboyish" ] ] text
     }

And for those of us who would like still more flexibility, the next versions should "import" all the Relax NG patterns in the Examplotron namespace so that we can use Relax NG compositors, patterns and name classes where needed.

!!Use scenarios

Why should we want to use Examplotron instead of Relax NG? We could reverse the question and ask why should we want to use Relax NG instead of Examplotron! At the end of the day, I think that it doesn't really matter. What's important is that the semantics of the validation engine is rock solid and has no limitations. As for the syntax which you'll be using to express your schemas, you can just use the one you like. And if you like the visual qualities of Examplotron, there is no reason to use any other: you will just be looking at a Relax NG schema under a different angle.

!!!Literate Programming

A common approach to software documentation include extracting documentation from the source documents relying on the structure of the programs and their comments (a good example is JavaDoc, the documentation extracting tool shipped with Java and u universally used on Java projects). Other project keep code and documentation separate and for both approaches, it is often the case that documentation and comments evolve separately from the code and that the documentation becomes sooner or later out of date.

The reason for this is that the focus is on the code and that documentation is often considered as a side product, less important than the code. Donald Knuth, the inventor of the term "literate programming", claims on the contrary:

"I believe that the time is ripe for significantly better documentation of programs, and that we can best achieve this by considering programs to be works of literature. Hence, my title: "Literate Programming."

Let us change our traditional attitude to the construction of programs: Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do.

The practitioner of literate programming can be regarded as an essayist, whose main concern is with exposition and excellence of style. Such an author, with thesaurus in hand, chooses the names of variables carefully and explains what each variable means. He or she strives for a program that is comprehensible because its concepts have been introduced in an order that is best for human understanding, using a mixture of formal and informal methods that reinforce each other."

(Donald Knuth. "Literate Programming (1984)" in Literate Programming. CSLI, 1992, pg. 99.)

Norm Walsh has adapted the concept to XML and tools for doing Literate Programming in XML are shipped under the name "litprog" by the DocBook project on SourceForge (http://sourceforge.net/projects/docbook/). The basic idea is to include snippet of code (or snippet of schemas in our case) within the documentation which can be written in any XML format including XHTML or DocBook. From this single document embedding code in documentation, a couple of XSLT transformation generate then a formated documentation and the source code.

Beyond the fact that you're working upside down compared to the common usage to add comments in the code, the other major practical difference is that you are now defining the relations between the snippets of code or schema using the mechanisms of "litprog" instead of using the mechanisms which are specific to each programming language. The granularity of the documentation becomes virtually independent of the granularity of your functions, methods, or in our case of the granularity of our named patterns. This has also the benefit, if needed, that we could group in a single literate documentation several languages and describe for instance the Relax NG schema of a document together with a XSLT transformation to manipulate the document and a DOM application to read it.

!!Out of the box

Literate Programming plays rather well with Relax NG as we will show on a small example. A literate programming document is a documentation embedding "src:fragment" elements to mark the fragments of schema which will assembled into complete schemas. The documentation can use any format such as DocBook, XHTML or even RDDL. Using XHTML, the description of the "name" element could be:

 <div>
 <h2>The <tt>name</tt> element</h2>
 <p>This is the name of the character.</p>
 <src:fragment id="name" xmlns="">
  <rng:element name="name">
    <rng:text/>
  </rng:element>
 </src:fragment>
 </div>

Or, with the compact syntax:

 <div>
 <h2>The <tt>name</tt> element</h2>
 <p>This is the name of the character.</p>
 <src:fragment id="name" xmlns="">
	element name { text }
 </src:fragment>
 </div>

In this first snippet, the definition of our element doesn't use any existing pattern, but a definition can also make a reference to a "src:fragment" element using "src:fragref", such as in:

 <div>
 <h1>The <tt>character</tt> element</h1>
 <p>The <tt>character</tt> element is the container holding all the information about a character.</p>
 <src:fragment id="character" xmlns="">
 <rng:element name="character">
  <src:fragref linkend="id"/>
  <src:fragref linkend="name"/>
  <src:fragref linkend="born"/>
  <rng:optional>
    <src:fragref linkend="qualification"/>
  </rng:optional>
 </rng:element>
 </src:fragment>
 </div>

Or, using the compact syntax:

 <div>
 <h1>The <tt>character</tt> element</h1>
 <p>The <tt>character</tt> element is the container holding all the information about a character.</p>
 <src:fragment id="character" xmlns="">
  element character {
    <src:fragref linkend="id"/>,
    <src:fragref linkend="name"/>,
    <src:fragref linkend="born"/>,
    <src:fragref linkend="qualification"/> ?
	}
 </src:fragment>
 </div>

From this literate programming document, two different outputs may be generated through XSLT transformations. The first one is the schema itself. Assuming we've defined all the sub-elements and attribute of our character element, the generated schema will be:

 <?xml version="1.0" encoding="utf-8"?>
 <rng:grammar xmlns:rng="http://relaxng.org/ns/structure/1.0"
   xmlns:src="http://nwalsh.com/xmlns/litprog/fragment"
   datatypeLibrary="http://www.w3.org/2001/XMLSchema-datatypes">
   <rng:start>
     <rng:element name="character">
       <rng:attribute name="id">
         <rng:data type="id" datatypeLibrary="http://relaxng.org/ns/compatibility/datatypes/1.0"/>
       </rng:attribute>
       <rng:element name="name">
         <rng:text/>
       </rng:element>
       <rng:element name="born">
         <rng:data type="date"/>
       </rng:element>
       <rng:optional>
         <element name="qualification">
           <text/>
         </element>
       </rng:optional>
     </rng:element>
   </rng:start>
 </rng:grammar>

Or, using the compact syntax:

 datatypes d = "http://relaxng.org/ns/compatibility/datatypes/1.0"
 start=
       element character {
  	attribute id { d:id },
     	element name { text },
     	element born { xsd:date },
     	element qualification { text } ?
 	}

This is a pretty normal schema and the thing we need to highlight is the way it has been modularized. Up to now, to split a schema into small and manageable pieces,  we've been using named patterns, a mechanism provided by Relax NG itself. We could have done so in our last example, but we have now a second possibility which is to use the mechanisms provides by the literate programming framework and define fragments and combine them using "src:fragment" and "src:fragref" instead of the "define" and "ref" elements from Relax NG. By doing so, we have generated a monolithic Russian doll schema through a modular description of its elements and attributes.

The second output from this literate programming document is the XHTML documentation:

[http://localhost/rngbook/literate-xml.png]

or, for the compact syntax:

[http://localhost/rngbook/literate-cpt.png]

!!Adding bells and whistles for RDDL

We have already mentioned RDDL in "Chapter 13: Annotating Schemas". RDDL (see http://rddl.org for more details) can be seen as a compromise to standardize a usage of XHTML and XLink that can be read both as plain XHTML by human beings with a standard web browser and by applications which will use the semantic attributes of XLink to discover resources such as schemas and stylesheets. 

RDDL documents can be generated from annotated Relax NG schema. When documenting XML vocabularies, the opposite works very well too and it is very tempting to use the literate programming framework to produce RDDL documents. There is no basic difference compared to what we've seen with XHTML and we could use the DocBook litprog stylesheets right away but we can also import them into stylesheets which will facilitate the authoring of RDDL documents.

RDDL is basically a compromise to share the same document between human readers through XHTML and applications through RDDL and XLink and the main burden when writing RDDL documents is that the information made available for the application needs to be repeated for the readers. For instance, to publish the snippet of schema describing the name element as a RDDL normative reference, we could write (note the "exclude-result-prefixes" and "mundane-result-prefixes" attributes which become needed to control various namespaces introduced for RDDL):

 <rddl:resource id="name-elt" xlink:type="simple"
                xlink:arcrole="http://www.rddl.org/purposes#normative-reference"
                xlink:role="http://www.w3.org/1999/xhtml"
                xlink:title="The name element"
                xlink:href="#name-elt">
 <div class="resource">
 <h2>The <tt>name</tt> element</h2>
 <src:fragment id="name" xmlns=""
               exclude-result-prefixes="cr xlink rddl rng"
               mundane-result-prefixes="rng">
  <rng:element name="name">
    <rng:text/>
  </rng:element>
 </src:fragment>
 </div>
 </rddl:resource>

That's not really tough, but there are some repetitions here: the content of the "h2" element is copied into "xlink:title" and xlink:href is constructed after the id attribute because the resource is local. For external resources, different but similar redundancies may be found too. When the RDDL document is generated by a XSLT transformation like it's the case with Literate Programming, it's tempting to define shortcuts that avoid these redundancies and write for instance:

 <cr:resource id="name-elt"
              role="http://www.w3.org/1999/xhtml"
              arcrole="http://www.rddl.org/purposes#normative-reference">
 <h2>The <tt>name</tt> element</h2>
 <p>This is the name of the character.</p>
 <src:fragment id="name" xmlns=""
               exclude-result-prefixes="cr xlink rddl rng"
               mundane-result-prefixes="rng">
  <rng:element name="name">
    <rng:text/>
  </rng:element>
 </src:fragment>
 </cr:resource>

Other features can easily be added, such as numbering the divisions, generating table of contents and indexes of resources and pretty printing code snippets and our document becomes:

[http://localhost/rngbook/literate-rddl.png]

!!!UML

UML (Unified Modeling Language) is an OMG standards which can be seen as the successor of the many Object Oriented methods invented in the 80s and 90s. The idea of using UML to model XML document is not new and good stuff has already been published on the subject (see for instance the book "Modeling XML applications with UML" by David Carlson published by Addison Wesley or his articles on XML.com).

There are two different levels at which UML and XML can be mapped.

*UML can be used to model the structure of XML documents directly. In this case, XML schemas can be generated for the purpose of validating the documents but they are provided as a convenience for the application more than a main delivery. Their style and modularity is not important as such and the algorithm to produce these schemas is focused on expressing validation rules as close as possible to the UML diagram.
*UML can be used to model a XML schema. In this case, the UML diagram is a higher level view of the schema and the schema by itself is the main delivery. The UML diagram needs to be able to control exactly how each schema structure is described and specific stereotypes and parameters are often added to provide the right level of control.

One of the points which appear quite clearly in all the work related to this topic is that it is quite easy to map UML objects into XML or, which is quite equivalent, to use UML to describe classes of instance documents. The most difficult issue when doing so is that UML is a graph while XML is a tree and some links need to be either removed or serialized using techniques such as XLink which are not built-in within XML 1.0. Except for this issue, the relationship between UML and XML is quite natural in both directions: UML provides a simple language to model XML documents and XML provides a natural serialization syntax for UML objects.

The other point which is also very clear is that it's not that simple to generate DTDs and W3C XML Schema schemas from UML.

The issue when generating DTDs or W3C XML Schemas from UML is to cope with the multiple restrictions of these languages, starting with those related to unordered content models: unordered content models are natural for UML for which the attributes of a class are unordered and the limitations of DTDs and W3C XML Schemas are a problem when UML attributes are serialized as XML elements.

The issue when modeling W3C XML Schema schemas is that the model does not only need to describe the XML instances but also the schema itself and all the complexity of W3C XML Schema kind of enters into UML world.

The situation with UML, XML and W3C XML Schema is that if there is a good overlap between UML and XML, the overlap is not so good between XML and W3C XML Schema and W3C XML Schema adds to XML its own concepts. To draw a rough picture, the situation is not unlike:

[http://localhost/rngbook/overlap.png]

With Relax NG, on the contrary, the overlap between XML and the schema language is near to perfection: Relax NG can describe almost any XML structure and having no notion of Post Schema Validation Infoset (PSVI), Relax NG doesn't add anything to XML. The overlap between UML, XML and Relax NG is thus almost as big as the overlap between UML and XML:

[http://localhost/rngbook/overlap2.png]

Designed with a UML editor such as ArgoUML, our library could be described as:

[http://localhost/rngbook/argouml.gif]

Note that I have been using conventions which look natural but are far from being official. For instance, I have prefixed the attribute names with "@", an idea borrowed to Will Provost on XML.com. Also, to model the title element with its text node and attribute, I have used the name "rng:data" to name its text content as a UML attribute.

ArgoUML saves its documents using the XML Metadada Interchange (XMI) format defined by the Object Management Group (OMG). You'll find more information about XMI at http://www.omg.org/technology/xml/.

XMI is pretty verbose and the XMI document generated by ArgoUML for this diagram is more than 800 lines long. I won't include it here, but there is no major difficulty to generate from this document a schema with unordered content models, such as:

  <?xml version="1.0" encoding="utf-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0"
          datatypeLibrary="http://www.w3.org/2001/XMLSchema-datatypes">
   <start>
     <element name="library">
       <interleave>
         <zeroOrMore>
           <element name="book">
             <interleave>
               <element name="isbn">
                 <data type="token"/>
               </element>
               <attribute name="id">
                 <data type="token"/>
               </attribute>
               <attribute name="available">
                 <data type="boolean"/>
               </attribute>
               <zeroOrMore>
                 <element name="author">
                   <interleave>
                     <attribute name="id">Foundation.Data_Types.Multiplicity
                       <data type="token"/>
                     </attribute>
                     <element name="name">
                       <data type="token"/>
                     </element>
                     <element name="born">
                       <data type="date"/>
                     </element>
                     <element name="died">
                       <data type="date"/>
                     </element>
                   </interleave>Foundation.Data_Types.Multiplicity
                 </element>
               </zeroOrMore>
               <zeroOrMore>Foundation.Data_Types.Multiplicity
                 <element name="character">
                   <interleave>
                     <attribute name="id">
                       <data type="token"/>
                     </attribute>
                     <element name="name">
                       <data type="token"/>
                     </element>
                     <element name="born">
                       <data type="date"/>
                     </element>
                     <element name="qualification">
                       <data type="token"/>
                     </element>
                   </interleave>
                 </element>
               </zeroOrMore>
               <element name="title">
                 <attribute name="xml:lang">
                   <data type="language"/>
                 </attribute>
                 <data type="token"/>
               </element>
             </interleave>
           </element>
         </zeroOrMore>
       </interleave>
     </element>
   </start>
 </grammar>

Or, after conversion by Trang:

 start =
   element library {
     element book {
       element isbn { xsd:token }
       & attribute id { xsd:token }
       & attribute available { xsd:boolean }
       & element author {
           attribute id { xsd:token }
           & element name { xsd:token }
           & element born { xsd:date }
           & element died { xsd:date }
         }*
       & element character {
           attribute id { xsd:token }
           & element name { xsd:token }
           & element born { xsd:date }
           & element qualification { xsd:token }
         }*
       & element title {
           attribute xml:lang { xsd:language },
           xsd:token
         }
     }*
   }

In fact, the only trouble I have had with Relax NG itself comes out of one of the few restrictions of Relax NG which we've mentioned in "Chapter 7: Constraining Text Values": data pattern cannot be interleaved and when generating this schema one must be careful to treat complex type simple content models (i.e. elements such as the "title" element which accepts attributes and text nodes but no children elements) as an exception. This straight translation is, of course, impossible with W3C XML Schema because of the cardinality of the character and author sub elements and containers would need to be added to fit into the limitations of the language.

Note that here we have generated a Russian doll design but that depending on the strategy used in the translation, we could have generated other "flavors" as well.

!!!Spreadsheets

The last transformation I'd like to show here is probably much widely used than one would think. Spreadsheets are very convenient to store and manipulate large lists of information items and have been used as a modeling tool for many years. This has been recently acknowledged by the UBL Oasis Technical committee (see http://www.oasis-open.org/committees/ubl/) which is in charge of a set of "core components" to be used by B2B applications and frameworks such as ebXML. Although this project is using a UML methodology, the release note of their 0.70 version states: "The current spreadsheet matrix used by UBL has proved the most versatile and manageable in developing a logical model of the UBL Library."

Now most if not all of the spreadsheet software are supporting XML format, generating Relax NG schemas from such a tool is really easy. There is no standard ways of representing XML documents in a spreadsheet. The benefit of spreadsheets is their flexibility and we can define specific layouts specific to each application. Coming back to our library, we could formalize it as:

[http://localhost/rngbook/oo.png]

That's basically nothing more than than a catalog of each information item with just enough information to generate a schema. The benefit of getting it as a spreadsheet is that it's easy to read and, when the catalog gets bigger, features such as filters, sort and search become increasingly useful to navigate over the catalog.

Again, generating Relax NG schemas is really easy and it doesn't take long before you turn this spreadsheet into schemas such as:

 <?xml version="1.0" encoding="utf-8"?>
 <grammar xmlns="http://relaxng.org/ns/structure/1.0"
          xmlns:a="http://relaxng.org/ns/compatibility/annotations/1.0"
          datatypeLibrary="http://www.w3.org/2001/XMLSchema-datatypes">
   <start>
     <element name="library">
       <a:documentation>Root element. Describes the whole library.</a:documentation>
       <zeroOrMore>
         <element name="book">
           <a:documentation>Describes a book.</a:documentation>
           <attribute name="id">
             <a:documentation>Identifier</a:documentation>
             <data type="token"/>
           </attribute>
           <attribute name="available">
             <a:documentation>Is the book available?</a:documentation>
             <data type="boolean"/>
           </attribute>
           <element name="isbn">
             <a:documentation>ISBN number</a:documentation>
             <data type="token"/>
           </element>
           <element name="title">
             <a:documentation>Title of the book</a:documentation>
             <data type="token"/>
             <attribute name="xml:lang">
               <a:documentation>Language</a:documentation>
               <data type="language"/>
             </attribute>
           </element>
           <zeroOrMore>
             <element name="author">
               <a:documentation>Author of a book</a:documentation>
               <attribute name="id">
                 <a:documentation>Identifier</a:documentation>
                 <data type="token"/>
               </attribute>
               <element name="name">
                 <a:documentation>Name</a:documentation>
                 <data type="token"/>
               </element>
               <element name="born">
                 <a:documentation>Date of birth</a:documentation>
                 <data type="date"/>
               </element>
               <element name="died">
                 <a:documentation>Date of death</a:documentation>
                 <data type="date"/>
               </element>
             </element>
           </zeroOrMore>
           <zeroOrMore>
             <element name="character">
               <a:documentation>Character of a book</a:documentation>
               <attribute name="id">
                 <a:documentation>Identifier</a:documentation>
                 <data type="token"/>
               </attribute>
               <element name="name">
                 <a:documentation>Name</a:documentation>
                 <data type="token"/>
               </element>
               <element name="born">
                 <a:documentation>Date of birth</a:documentation>
                 <data type="date"/>
               </element>
               <element name="qualification">
                 <a:documentation>Qualification of a character</a:documentation>
                 <data type="token"/>
               </element>
             </element>
           </zeroOrMore>
         </element>
       </zeroOrMore>
     </element>
   </start>
 </grammar>

or, after a translation by Trang:

 namespace a = "http://relaxng.org/ns/compatibility/annotations/1.0"

 start =
   ## Root element. Describes the whole library.
   element library {
     ## Describes a book.
     element book {
       ## Identifier
       attribute id { xsd:token },
       ## Is the book available?
       attribute available { xsd:boolean },
       ## ISBN number
       element isbn { xsd:token },
       ## Title of the book
       element title {
         xsd:token,
         ## Language
         attribute xml:lang { xsd:language }
       },
       ## Author of a book
       element author {
         ## Identifier
         attribute id { xsd:token },
         ## Name
         element name { xsd:token },
         ## Date of birth
         element born { xsd:date },
         ## Date of death
         element died { xsd:date }
       }*,
       ## Character of a book
       element character {
         ## Identifier
         attribute id { xsd:token },
         ## Name
         element name { xsd:token },
         ## Date of birth
         element born { xsd:date },
         ## Qualification of a character
         element qualification { xsd:token }
       }*
     }*
   }

And here again, instead of a Russian doll schema, we could have generated any other style of schema.
