So far, we have only described ordered groups of elements and text nodes and we will now see other patterns describing choices and unordered sequences. Although this class of patterns have no special name in the Relax NG specification, we will call them "compositors" in this book by analogy with the compositors defined by W3C XML Schema and because this name is accurate to describe these patterns which are used to compose complex patterns out of the basic patterns matching actual nodes in the document such as "element", "attribute" and "text". One of the key differentiators between these compositors and patterns matching nodes is that compositors are syntactical elements which have no existence out of a schema and we will insist on this aspect since it is easy to forget it when we focus on a schema and kind of forget the instance document and this may lead to unexpected errors.

!!The group pattern

When we have written, to define our "character" element:

     <element name="character">
      <attribute name="id"/>
      <element name="name">
       <text/>
      </element>
      <element name="born">
       <text/>
      </element>
      <element name="qualification">
       <text/>
      </element>
     </element>

we have not specified how the different nodes composing the "character" element needed to be composed and Relax NG has implied that we have been using a "group" compositor. This "group" compositor which is implied in the XML syntax is visible in the compact syntax where it is marked by the commas (",") used between the patterns:

   element character {
    attribute id {text},
    element name {text},
    element born {text},
    element qualification {text}}

When using the XML syntax, the "group" compositor may be explicitly specified and the above definition is strictly equivalent to:

     <element name="character">
      <group>
       <attribute name="id"/>
       <element name="name">
        <text/>
       </element>
       <element name="born">
        <text/>
       </element>
       <element name="qualification">
        <text/>
       </element>
      </group>
     </element>

Because the order of attributes is considered as not significant per XML 1.0, the semantic of the "group" compositor is slightly less straightforward than it appears and this compositor means something such as: "check that the patterns included in this compositor appear in the specified order except for attributes which may appear in any order in the start tag independently of the location where they are found in the patterns."

A last thing to keep in mind about "group" compositors is that as we have said about compositors in general there is no such thing as a "group of nodes" in an instance document and that the notion of "group" is a pattern which belongs only to our schema. One of the consequences of this statement is that there is no hard border to isolate nodes within a group from nodes out of the group and we will see in the section named "Why is it called "interleave" instead of "unorderedGroup"?" that in certain conditions nodes matching patterns defined outside of a group can be "inserted" in the group.

!!The "interleave" pattern

The second compositor, named "interleave", describes a set of unordered patterns, i.e. a set of patterns which will be validated if they match the content of the instance documents in any order. As far as validation is concerned, this behavior is similar to the validation of attributes in a "group" compositor up to the point that the algorithms to validate attributes within groups is the same than the algorithm to validate any node in "interleave" compositors. This is, of course, for validation purposes only and using "interleave" patterns do not imply that the order of elements and text nodes in the instance document will not be reported to the application but only that they are allowed to appear in any order.

To specify that "character" elements may accept children elements in any order, we just need to replace our explicit or implicit group pattern by an "interleave" pattern:

     <element name="character">
      <interleave>
       <attribute name="id"/>
       <element name="name">
        <text/>
       </element>
       <element name="born">
        <text/>
       </element>
       <element name="qualification">
        <text/>
       </element>
      </interleave>
     </element>

In the compact syntax, "interleave" patterns are marked using a ampersand ("&") character as a separator instead of the comma (",") which is the mark of ordered groups:

   element character {
    attribute id {text}&
    element name {text}&
    element born {text}&
    element qualification {text}}

These two equivalent schemas will validate "character" elements with children elements in any order, such as:

   <character id="PP">
    <name>Peppermint Patty</name>
    <born>1966-08-22</born>
    <qualification>bold, brash and tomboyish</qualification>
    </character>
   <character id="Snoopy">
    <born>1950-10-04</born>
    <qualification>extroverted beagle</qualification>
    <name>Snoopy</name>
   </character>
   <character id="Schroeder">
    <qualification>brought classical music to the Peanuts strip</qualification>
    <name>Schroeder</name>
    <born>1951-05-30</born>
   </character>

Although "interleave" looks straightforward at this level of description, we will see more of its complete behavior and restrictions in the last sections of this chapter. You can skip them if they look too complex for now but you should remember to come back and revisit them if your "interleave" patterns show unexpected results or error messages!

!!The "choice" pattern

Let's say that we want to add some flexibility to the "name" element and still accept:

    <name>Lucy</name>

but also:

    <name>
     <first>Charles</first>
     <middle>M</middle>
     <last>Schulz</last>
    </name>

and:

    <name>
     <first>Peppermint</first>
     <last>Patty</last>
    </name>

To express this, we will use a "choice" pattern which will accept either a text node or a group of three elements (one of them being optional):

   <element name="name">
    <choice>
     <text/>
     <group>
      <element name="first"><text/></element>
      <optional>
        <element name="middle"><text/></element>
      </optional>
      <element name="last"><text/></element>
     </group>
    </choice>
   </element>

The compact syntax uses a logical or character ("|") to mark choices:

 element name {
  text|(
   element first{text},
   element middle{text}?,
   element last{text}
  )}

Note that we had to use parenthesis "(..)" to mark the boundary of the "group" pattern which is one of the two alternatives of our "choice" pattern.

!!Pattern compositions

In the preceding example we have combined a choice with a group. This process can be general generalized and there is virtually no restriction nor limit in the way compositors can be combined. As an example, let's say we want now in our character element to allow either a "name" element or the three elements "first-name", "middle-name" (optional) and "last-name" in any relative order but before the "born" and "qualification" elements. That's no problem with Relax NG and we can just write:


     <element name="character">
      <attribute name="id"/>
      <choice>
       <element name="name"><text/></element>
       <interleave>
        <element name="first-name"><text/></element>
        <optional>
         <element name="middle-name"><text/></element>
        </optional>
        <element name="last-name"><text/></element>
       </interleave>
      </choice>
      <element name="born"><text/></element>
      <element name="qualification"><text/></element>
     </element>

or, with the compact syntax:

   element character {
    attribute id {text},
    (
     element name {text}|
     (
      element first-name {text}&
      element middle-name {text} ? &
      element last-name {text}
     )
    )

Note that we have added two levels of parenthesis here. This was needed because for the compact syntax, operators are used to determine the nature of compositors ("group", "interleave" or "choice"). They cannot be mixed at the same level and we need to use these parenthesis to explicitly mark where each compositor starts and ends.

These schemas will validate character elements such as:

   <character id="PP">
    <first-name>Peppermint</first-name>
    <last-name>Patty</last-name>
    <born>1966-08-22</born>
    <qualification>bold, brash and tomboyish</qualification>
   </character>

   <character id="PP2">
    <last-name>Patty</last-name>
    <first-name>Peppermint</first-name>
    <born>1966-08-22</born>
    <qualification>bold, brash and tomboyish</qualification>
   </character>

   <character id="Snoopy">
    <name>Snoopy</name>
    <born>1950-10-04</born>
    <qualification>extroverted beagle</qualification>
   </character>

   <character id="Snoopy2">
    <first-name>Snoopy</first-name>
    <middle-name>the</middle-name>
    <last-name>Dog</last-name>
    <born>1950-10-04</born>
    <qualification>extroverted beagle</qualification>
   </character>

or
   <character id="Snoopy3">
    <middle-name>the</middle-name>
    <last-name>Dog</last-name>
    <first-name>Snoopy</first-name>
    <born>1950-10-04</born>
    <qualification>extroverted beagle</qualification>
   </character>

The flexibility and freedom with which patterns can be combined and the lack of restrictions associated with these combinations is one of the main assets of Relax NG and a major differentiator with other XML schema languages.

!!The lack of order in a schema may be a source of information in instances

Before we move on to text patterns and mixed contents, let's think again on the "interleave" pattern. As we have already noted, the fact that these content models are often called "unordered" is misleading since although no ordered is required by the schema, the nodes will be ordered in the instance documents and the order in which they will appear in the document may be considered as significant by the applications. Going back to our example with the names, any application managing names probably needs to know what's my first name and what's my last name and with little additional effort they can get the information whether the first name comes before or after the last name in a XML document. The most user friendly of these applications may also want to know whether I prefer to be called with my first or last name first. Should we add an information item to carry this information when we can just rely on the order of these elements in the instance document?

In other words, defining a content using "interleave" patterns shouldn't be seen as a "degradation" of a schema to make the life of  document authors easier. Incidentally it does make their life easier and there is no reason to add arbitrary restrictions when there are not needed by the application but that's not the point I want to make here. The point I want to make here is that a content model defined with "interleave" patterns allow more combinations than those using "group" patterns and that they can be used to bring more information.

The downside with "interleave" patterns is that the freedom with which they can be used is unfortunately specific to Relax NG and if you need to insure that it will also be possible to model your vocabulary with a more rigid schema language such as W3C XML Schema, you will often have to restrict the usage of "interleave" patterns in your Relax NG schemas.

!!Text and empty patterns, whitespaces and mixed contents

So far, we have only used "text" patterns within "group" patterns and we've missed an important "detail" about this pattern which doesn't mean as one could have expected "a text node" but rather "zero or more text nodes" and this deserves some explanations. First, the reason why "text" patterns accept zero text nodes is linked to the policy adopted by Relax NG regarding whitespaces. Generally speaking, whitespace processing is somewhat fuzzy in XML and Relax NG has attempted to find a the "least surprising" policy to meet most common usages. We will see more whitespace processing when we will cover datatypes where they become significant, but right now, let's say that Relax NG doesn't make any difference between empty strings, no string at all, a string containing only whitespaces before or after an element node and in a lesser extend a single text child element containing only whitespaces.

For instance, in the following snippet:

 <foo at1="" at2=" ">
  <bar/>
  <bar></bar>
  <bar>
   <baz/>
   <baz/>
  </bar>
  <bar>
  </bar>
 </foo>

Relax NG considers that the values of at1 and at2, the content of the first and second "bar" elements, the text between the third "bar" start tag and the first "baz" element, the text between the two "baz" element and even the text within the last "bar" element is not significant and that they should be matching both "text" and "empty" patterns. Even though this is not obvious to explain and understand, that rule has been created to meet the most common practices in XML and you would probably never have noticed it if I had not insisted on it. Its two visible consequences for the patterns which we've seen so far are:

* Since "text" patterns match any text node they must match strings which are either empty or containing only whitespaces and since there is no difference between empty strings and no string, "text" patterns match "zero strings" i.e. they are always optional.
* Since "empty" patterns match "zero strings" and since there is no difference between no string and empty strings or strings containing only whitespaces, "empty" patterns match also strings either empty or containing only whitespaces.

In other words, the snippet shown above would match both content models where all the occurrences mentioned are described as "text" or "empty" patterns. And if we add the rule -already used a lot but not yet explicated- that says that you don't need to explicitly express "empty" patterns between elements, these two schemas would both validate this instance document:

 <element xmlns="http://relaxng.org/ns/structure/1.0" name="foo">
  <attribute name="at1"><text/></attribute>
  <attribute name="at2"><text/></attribute>
  <oneOrMore>
   <element name="bar">
    <choice>
     <text/>
     <oneOrMore>
      <element name="baz"><text/></element>
     </oneOrMore>
    </choice>
   </element>
  </oneOrMore>
 </element>

or

 <element xmlns="http://relaxng.org/ns/structure/1.0" name="foo">
  <attribute name="at1"><empty/></attribute>
  <attribute name="at2"><empty/></attribute>
  <oneOrMore>
   <element name="bar">
    <choice>
     <empty/>
     <oneOrMore>
      <element name="baz"><empty/></element>
     </oneOrMore>
    </choice>
   </element>
  </oneOrMore>
 </element>

After having seen why text patterns had to be optional, we need to see why it's useful for them to match multiple instances. The first point to note is that when a "text" pattern is used with a "group" or "choice" pattern this doesn't make any difference since text nodes are merged when they are contiguous or separated by infoset items not checked by Relax NG such as comments or Processing Instructions (PIs). Within a "group" or a "choice", there is thus no difference between a pattern which would match one or one or more text nodes. The only place where it can make a difference is thus within "interleave" compositors and that's the reason why this specificity has been introduced. Document oriented applications including XHTML provide numerous examples of so called mixed content elements which accept text end embedded elements in any order and in this case it would have no sense to limit the number of text nodes.

To introduce such a content model, let's say we want to extend the "title" element to include zero or more links using "a" elements with "href" attributes, such as:

   <title xml:lang="en">Being a
    <a href="http://dmoz.org/Recreation/Pets/Dogs/">Dog</a>
    Is a Full-Time
    <a href="http://dmoz.org/Business/Employment/Job_Search/">Job</a>
   </title>

The content of the new "title" element can be described as an "interleave" pattern allowing zero or more "a" elements and zero or more text nodes and the fact that the "text" pattern matches zero or more text nodes will avoid us to specify its cardinality and we will just write:

    <element name="title">
     <interleave>
      <attribute name="xml:lang"/>
      <zeroOrMore>
       <element name="a">
        <attribute name="href"/>
        <text/>
       </element>
      </zeroOrMore>
      <text/>
     </interleave>
    </element>

or, using the compact syntax:

   element title {
    attribute xml:lang {text}&
    element a {attribute href {text}, text}*&
    text
  }

Considering that this was still too verbose for something quite common, Relax NG has introduced a specific compositor named "mixed" which has the same meaning than "interleave including a text pattern" and these schemas are strictly equivalent to:

    <element name="title">
     <mixed>
      <attribute name="xml:lang"/>
      <zeroOrMore>
       <element name="a">
        <attribute name="href"/>
        <text/>
       </element>
      </zeroOrMore>
     </mixed>
    </element>

The "mixed" compositor is marked using a "mixed" keyword in the compact syntax and would be written as:

 element title {
  mixed {
   attribute xml:lang {text}&
   element a {attribute href {text}, text} *
  }
 }

!!Why is it called "interleave" instead of "unorderedGroup"?

If "interleave" was only about defining unordered groups why would it be called "interleave" and not "unorderedGroup" or something similar? As you'll have guessed, there is something else hidden behind this name and interleave is not only a definition of unordered groups, but a definition of unordered groups which let their child nodes intermix within subgroups even when these groups are ordered groups. Don't worry, this concept is simpler than it looks when we try to give a semi-formal definition and an example will make it easy to grasp!

This behavior of ordered groups immersed in an unordered may be surprising and we can try a real world metaphor to illustrate it. If we imagine that the leaf nodes of a XML document are like a bunch of tourists visiting a museum, we can define the unordered sets of all the tourists visiting and ordered groups of tourists following their guides. There are different ways to immerse ordered groups within the unordered set of the museum visitors and to mix ordered groups together and the interleave pattern describes one specific way to do this immersion: when the museum is an interleave pattern, the ordered groups only preserve the relative order of their members and not only allow individual tourists to insert themselves within a group but also two groups to "interleave" their members.

To come back to XML and Relax NG, let's take the following schema:

 <element xmlns="http://relaxng.org/ns/structure/1.0" name="museum">
  <interleave>
   <element name="individual"><empty/></element>
   <group>
    <element name="group-member1"><empty/></element>
    <element name="group-member2"><empty/></element>
   </group>
  </interleave>
 </element>

or, using the compact syntax:

 element museum {
  element individual {empty} &
  (
   element group-member1 {empty},
   element group-member2 {empty}
  )
 }

Element "individual" is an individual visiting the museum and elements "group-member1" and "group-member2" are in a group. Because "interleave" patterns are non ordered groups, the following instance documents are valid:

 <museum>
  <individual/>
  <group-member1/>
  <group-member2/>
 </museum>

and

 <museum>
  <group-member1/>
  <group-member2/>
  <individual/>
 </museum>

These documents are instances where the element "individual" which matches the first pattern in the interleave pattern (i.e. the element pattern) is either before or after the elements "group-member1" and "group-member2" which match the "group" pattern which is the second sub-pattern of the "interleave" pattern. And, because the "interleave" pattern allows that the nodes matching its sub-pattern are mixed, the schema also validate this third combination:

 <museum>
  <group-member1/>
  <individual/>
  <group-member2/>
 </museum>

On the other hand, all the combinations where the relative order between group members would not be respected would be invalid. A example of such invalid combinations is:

 <museum>
  <group-member2/>
  <individual/>
  <group-member1/>
 </museum>

Interleave can also be used to "mix" two groups of patterns and in this case the relative order of the element of each group is maintained but the groups may elements of different groups may appear in any order and the groups may be "interleaved". For instance, the following schema:

 <element xmlns="http://relaxng.org/ns/structure/1.0" name="museum">
  <interleave>
   <group>
    <element name="group1.member1"><empty/></element>
    <element name="group1.member2"><empty/></element>
   </group>
   <group>
    <element name="group2.member1"><empty/></element>
    <element name="group2.member2"><empty/></element>
   </group>
  </interleave>
 </element>

or, using the compact syntax:

 element museum{
  (
   element group1.member1 {empty},
   element group1.member2 {empty}
  ) & (
   element group2.member1 {empty},
   element group2.member2 {empty}
  )
 }

will validate documents such as:

 <museum>
  <group1.member1/>
  <group1.member2/>
  <group2.member1/>
  <group2.member2/>
 </museum>

and

 <museum>
  <group2.member1/>
  <group2.member2/>
  <group1.member1/>
  <group1.member2/>
 </museum>

where the groups are kept separated but also:

 <museum>
  <group1.member1/>
  <group2.member1/>
  <group2.member2/>
  <group1.member2/>
 </museum>

or

 <museum>
  <group1.member1/>
  <group2.member1/>
  <group1.member2/>
  <group2.member2/>
 </museum>

where the groups are interleaved.

!!Ordered mixed content models

We have seen that an pattern interleaved with a group is allowed to appear anywhere between the patterns of the group and this feature may be used with a text pattern to define ordered mixed content models where the text nodes may appear anywhere but the order of the elements is fixed. These content models are quite uncommon in XML and a use case could be a data oriented vocabulary such as our library in which optional text could be inserted to provide more user friendly document such as:

   <character id="Lucy">
    <name>Lucy</name> made her first apparition in a Peanuts strip on
    <born>1952-03-03</born>, and the least we can say about her is that she is
    <qualification>bossy, crabby and selfish</qualification>.
   </character>

If we want to fix the order of the children elements, we can just embed a "group" pattern inside a "mixed" pattern:

     <element name="character">
      <mixed>
       <attribute name="id"/>
       <group>
        <element name="name">
          <text/>
        </element>
        <element name="born">
          <text/>
        </element>
        <element name="qualification">
         <text/>
        </element>
       </group>
      </mixed>
     </element>

Per the definition of the "mixed pattern", this is equivalent to:

     <element name="character">
      <interleave>
       <attribute name="id"/>
       <text/>
       <group>
        <element name="name">
          <text/>
        </element>
        <element name="born">
          <text/>
        </element>
        <element name="qualification">
         <text/>
        </element>
       </group>
      </interleave>
     </element>

The "text" pattern matches text nodes before, after or between the elements of the group but as we've seen with our museum example in the previous section, the order of the elements in the group will be enforced. The compact syntax will use "mixed" keyword with commas (",") between sub-patterns to express this:

   element character {
    mixed {
     attribute id {text},
     element name {text},
     element born {text},
     element qualification {text}
    }
   }

We have already seen that the compact syntax "mixed" keyword can be used using ampersands ("&") and commas (",") to define unordered and ordered mixed patterns and to be exhaustive, we must mention that or ("|") can also be used to interleave text nodes in "choice" patterns:

 element foo{
  mixed {
   (
    element in1.1 {empty},
    element in1.2 {empty}
   ) | (
    element in2.1 {empty}&
    element in2.2 {empty}
   )
  }
 }

This pattern is interleaving text nodes into either a group of in1.1 and in1.2 elements or an interleave pattern of elements in2.1 and in2.2. In the first case because of the semantic of "group" patterns the order between elements is fixed while in the second one  the order doesn't matter. Mixed choice contents do not constitute new content models and are equivalent to choices of mixed content models: we could rewrite this schema as:


 element foo{
  (
   mixed{
    element in1.1 {empty},
    element in1.2 {empty}
    }
  ) | (
   mixed{
    element in2.1 {empty}&
    element in2.2 {empty}
   }
  )
 }
 }

!!Principal restriction related to "interleave"

We will see the different restrictions of Relax NG in "Chapter 15: Simplification And Restrictions", but we need to mention the principal restriction related to the "interleave" compositor which will probably bite you at some point if you combine mixed content models.

Let's say we want to extend our "title" element to allow not only links ("a" elements) but also bold characters marked by a "b" element:

   <title xml:lang="en">Being a
    <a href="http://dmoz.org/Recreation/Pets/Dogs/">Dog</a>
    Is a <b>Full-Time</b>
    <a href="http://dmoz.org/Business/Employment/Job_Search/">Job</a>
   </title>

Because text may appear before the "a" elements, between "a" and "b" and after the "b" element, we might be tempted to write the following schema:

 <element name="title">
   <interleave>
     <attribute name="xml:lang"/>
     <text/>
     <zeroOrMore>
       <element name="a">
         <attribute name="href"/>
         <text/>
       </element>
     </zeroOrMore>
     <text/>
     <zeroOrMore>
       <element name="b">
         <text/>
       </element>
     </zeroOrMore>
     <text/>
   </interleave>
 </element>

or:

 element title {
   attribute xml:lang {text}
   & text
   & element a {attribute href {text}, text} *
   & text
   & element b {text} *
   & text
 }

Running jing against this schema will raise the following error:

  Error at URL "file:/home/vdv/xmlschemata-cvs/books/relaxng/examples/RngMorePatterns/interleave-restriction2.rnc",
  line number 1, column number 2: both operands of "interleave" contain "text"

This is because there can be only one "text" pattern in each "interleave" pattern. We have seen that "text" patterns match zero or more text nodes, and in this case, the remedy is simple enough; the schema must be rewritten to:

 <element name="title">
   <interleave>
     <attribute name="xml:lang"/>
     <text/>
     <zeroOrMore>
       <element name="a">
         <attribute name="href"/>
         <text/>
       </element>
     </zeroOrMore>
     <zeroOrMore>
       <element name="b">
         <text/>
       </element>
     </zeroOrMore>
   </interleave>
 </element>

Or:

 element title {
   attribute xml:lang {text}
   & text
   & element a {attribute href {text}, text} *
   & element b {text} *
 }

This new schema is perfectly valid and it does what we meant to do with our invalid schema.

On this simple example, the diagnostic has been very simple but in practice the situation is usually more complex with conflicting "text" patterns belonging to different sub patterns of "interleave" or "mixed". When using pattern libraries (as shown in "Chapter 10: Creating Building Blocks"), the conflicting "text" patterns will often belong to different Relax NG grammars, making it still tougher to pinpoint where the problem is. To make it still worse the error messages from the Relax NG processors are often quite cryptic, telling you that you have conflicting "text" patterns in "interleave" without saying where they come from and, unfortunately, you'll have to figure out by yourself.

The reason behind this restriction is to optimize Relax NG implementations using the derivative method described by James Clark. Instead of processing each text node when processing mixed content models these implementations can just memorize the fact that this is mixed content and ignore each text node. To do so, the implementation needs to be able to quickly find if a content model is mixed or not and that's where the restriction does make a difference in term of programming complexity and execution speed.

!!Missing pattern

We have seen that the "interleave" pattern associates two different features and is both an unordered group and something which alters the way sub-groups can be combined together. These two features are not totally independent since mixing child nodes only have a meaning when the order of the sub-groups is not maintained but they are not totally dependent either and in theory it would be possible to define a pattern with a meaning of "unordered group" which would not have the effect of interleaving child nodes and would keep groups unaltered.

If this pattern doesn't exist in Relax NG, this is not only to keep the language as simple as possible but also because although it is built on top of an abstract mathematical model, Relax NG is also built on top of the experience of its authors who have wanted to focus on general usages and best practices amongst the XML community and we must say that the lack of a "unordered group with no interleaving" hasn't been reported as a limitation with real world applications so far.
