relax-ng-textbook/src/docbook/ch16.xml

<?xml version="1.0" encoding="iso-8859-1"?>
<?xml-stylesheet type="text/css" href="chapter.css"?><chapter>
  <title>Chapter 16:    Determinism and Datatype Assignment</title>
  <simplesect>
    <para>One of the strengths of Relax NG is its flexibility in supporting scaring concepts called &quot;ambiguous content models&quot; (SGML world) &quot;non-deterministic content models&quot; (XML DTDs) or &quot;Unique Particle Attribution rule&quot;  and &quot;Consistent Declaration rule&quot; (W3C XML Schema). </para>
    <para>Before you read on into this chapter, let's make it clear that as far as validation only is concerned, it's perfectly fine with Relax NG to write ambiguous schemas. </para>
    <para>That being said, when type assignment or data binding is involved, ambiguity may become a problem and we will see in this chapter how Relax NG can be used to be &quot;type assignment friendly&quot;.</para>
  </simplesect>
  <sect1>
    <title>What are we talking about?</title>
    <para>The thing we need to do first is to try to clarified these notions which are blurred in many papers and discussions and are not as obscure as people often think.</para>
    <sect2>
      <title>Ambiguity versus determinism</title>
      <para>The first distinction to make is to differentiate what's called ambiguity (or rather unambiguity) and what's called determinism. These two terms have been given precise definitions by regular expressions and hedge grammars theoreticians and part of the confusion around these notions comes from the fact that they are often misused.</para>
      <para>A schema is said to be ambiguous when a document may be valid through different pattern alternatives. A trivial example is:</para>
      <programlisting>
<![CDATA[ element foo{empty} | element foo{empty}]]>
      </programlisting>
      <para>When an empty element named <literal>foo</literal> is found in an instance document, there is no way to say if it is valid per the left or right definition of &quot;element foo{empty}&quot; in the schema.</para>
      <para>There are, of course, more complex cases of ambiguity and we'll see some of them in the next sections, but this is the general idea behind ambiguity.</para>
      <para>Ambiguity (or unambiguity) is independent of any implementation or algorithm. It's a property of the schema itself and without rewriting it a schema is either ambiguous or not.</para>
      <para>On the contrary, determinism has been introduced to facilitate implementation of schema processors. The basic idea beyond determinism is that at each point when matching an instance document against a schema, the schema processor has at most one possible choice. This is making the life easier for implementers which can safely rely on well known algorithm such as automatons (also called Finite State Machines or FSMs) and be sure that their computation times will not grow up exponentially. This is also a major constraint imposed on schema authors.</para>
      <para>An ambiguous schema is always non deterministic, but the opposite is far from being true. Consider for instance:</para>
      <programlisting>
<![CDATA[ element foo{empty} | (element foo{empty}, element bar{empty})]]>
      </programlisting>
      <para>This is not ambiguous since after having read the element after an empty element named <literal>foo</literal> a schema processor is able to say if the right or left alternative is being used (or none if the document is invalid) but this is non deterministic since when a schema processor is matching an empty element named <literal>foo</literal> it has two different choices and cannot choose between them without looking ahead at the next element.</para>
      <para>Ambiguous schemas are not a problem as long as validation only is concerned: their validation reports are consistent and we don't care why a document is valid or not as long as the answer (valid or invalid) is reliable. The only real downside about ambiguous schemas is for applications performing datatype assignment (or more generally instance document annotation) through validation and we will see more about these issues in the next sections of this chapter.</para>
      <para>The main issue with schema languages requiring deterministic schemas is that some content models are fundamentally non deterministic and cannot be rewritten in a deterministic form. Such schema languages are not only adding restrictions on the forms to use to write a schema but their expressive power is limited and they cannot describe all the content models allowed in well formed XML. We will see examples of content models impossible to describe in a deterministic form in the section about compatibility with W3C XML Schema.</para>
    </sect2>
    <sect2>
      <title>Different types of ambiguities</title>
      <para>In a Relax NG schema, we can distinguish four different types of ambiguities: regular expression ambiguities, hedge grammar ambiguities, name classes ambiguity and datatype ambiguities and we'll briefly introduce them since they have slightly different behaviors.</para>
      <sect3>
        <title>Regular expression ambiguities</title>
        <para>Note that in this chapter we are using the term &quot;regular expression&quot; as used in  the math behind Relax NG. The term &quot;regular expression&quot; that you'll find in this chapter should thus not be confused with the regular expressions which we've seen in the W3C XML Schema <literal>pattern</literal> facet.</para>
        <para>After a schema has been simplified, we can make a clear distinction between the definition of each element (embedded in its own named pattern) and the grammar which combines these definitions. What's called a regular expression ambiguity is an ambiguity which resides within the definition of an element.</para>
        <para>Theoreticians have demonstrated that any ambiguous regular expressions may be rewritten in an unambiguous way and these ambiguities may be considered as unlucky variations over unambiguous schemas.</para>
        <para>A basic example of such a choice between a pattern and itself is:</para>
        <programlisting>
<![CDATA[ <choice>
  <ref name="pattern"/>
  <ref name="pattern"/>
 </choice>]]>
        </programlisting>
        <para>or:</para>
        <programlisting>
<![CDATA[ pattern|pattern]]>
        </programlisting>
        <para>Obvious in this case, the unambiguous form is more or less difficult to find when the ambiguous pattern gets more complex. For instance, the following pattern:</para>
        <programlisting>
<![CDATA[ <choice>
  <group>
   <optional>
    <ref name="first"/>
   </optional>
   <ref name="second"/>
  </group>
  <group>
   <ref name="second"/>
   <optional>
    <ref name="third"/>
   </optional>
  </group>
 </choice>]]>
        </programlisting>
        <para>or:</para>
        <programlisting>
<![CDATA[ (first?,second)|(second,third?)]]>
        </programlisting>
        <para>Is ambiguous because an instance nodeset matching only the named pattern <literal>second</literal> without the leading <literal>first</literal> nor the ending <literal>third</literal> is valid per the two alternative of the choice. It can be rewritten by removing the option of matching only the <literal>second</literal> pattern from one of the alternatives:</para>
        <para>or:</para>
        <programlisting>
<![CDATA[  <group>
   <optional>
    <ref name="first"/>
   </optional>
   <ref name="second"/>
  </group>
  <group>
   <ref name="second"/>
   <ref name="third"/>
  </group>
 </choice>]]>
        </programlisting>
        <para>or:</para>
        <programlisting>
<![CDATA[ (first?,second)|(second,third)]]>
        </programlisting>
        <para>Algorithms have been developed to rewrite ambiguous regular expressions in their unambiguous forms and it would be really useful if XML development tools could implement them to propose non ambiguous alternatives for ambiguous patterns when they exist. Until this happens, the best thing to do when you are confronted with an ambiguous pattern to disambiguate is to take a step back, grab a cup of tea or coffee and calmly write the different combinations expressed by the schema to combine them differently till the combination isn't ambiguous any longer.</para>
        <para>Note that explicit choices aren't the only pattern which can lead to ambiguous schemas. Consider this simple pattern:</para>
        <programlisting>
<![CDATA[ <group>
  <optional>
   <ref name="pattern"/>
  </optional>
  <optional>
   <ref name="pattern"/>
  </optional>
 <group>]]>
        </programlisting>
        <para>or:</para>
        <programlisting>
<![CDATA[ pattern?, pattern?]]>
        </programlisting>
        <para>If we have a content model which matches only one pattern, we cannot know if it will match it for the first or the second occurrence of the pattern and this schema can be considered as ambiguous. To rewrite it as a non ambiguous schema, we could write:</para>
        <programlisting>
<![CDATA[  <optional>
   <ref name="pattern"/>
   <optional>
    <ref name="pattern"/>
   </optional>
  </optional>]]>
        </programlisting>
        <para>or:</para>
        <programlisting>
<![CDATA[ (pattern, pattern?)?]]>
        </programlisting>
        <para>Although the way leading to rewritings may look opaque, the math behind Relax NG can help us like high school algebra helps us factorize mathematical expressions. As an exercise, let's decompose the chain of factorizations and simplifications to rewrite &quot;pattern?, pattern?&quot; as &quot;(pattern, pattern?)?&quot;.</para>
        <para>The first step relies on the fact that &quot;pattern?&quot; is equivalent to &quot;empty|pattern&quot;:</para>
        <programlisting>
<![CDATA[ pattern?, pattern?]]>
        </programlisting>
        <para>is equivalent to:</para>
        <programlisting>
<![CDATA[ (empty|pattern), (empty|pattern)]]>
        </programlisting>
        <para>which can be factorized as:</para>
        <programlisting>
<![CDATA[ (empty,empty)|(empty,pattern)|(pattern,empty)|(pattern,pattern)]]>
        </programlisting>
        <para>which can be simplified as:</para>
        <programlisting>
<![CDATA[ empty|pattern|(pattern,pattern)]]>
        </programlisting>
        <para>which is equivalent to:</para>
        <programlisting>
<![CDATA[ empty|(pattern,(empty|pattern))]]>
        </programlisting>
        <para>which is equivalent to</para>
        <programlisting>
<![CDATA[ (pattern, pattern?)?]]>
        </programlisting>
        <para>We could argue whether the unambiguous forms are clearer, more logical an easier to read than the ambiguous forms or not, but I think that the answer would be very subjective. These different forms are highly dependent of the perspective from which we have analyzed the content of instance documents. There isn't a good nor a bad form and working with a schema language such as Relax NG which supports all of these forms does save a lot of time: you don't have to take a perspective imposed by the language! </para>
        <para>A last thing to note is that disambiguating regular expressions does not significantly change the structure or the style of your schema since the changes are limited to the regular expression itself and this will not be the case of ambiguous regular hedge grammars.</para>
      </sect3>
      <sect3>
        <title>Ambiguous regular hedge grammars</title>
        <para>In the case of a Relax NG schema, we've defined a regular expression ambiguity as an ambiguity which resides within the definition of an element. Ambiguous regular hedge grammars are on the contrary ambiguities spread over element definitions which play the role of &quot;hedges&quot; in a Relax NG schema. A example of an ambiguous regular hedge grammar is:</para>
        <programlisting>
<![CDATA[    <choice>
      <ref name="pattern1"/>
      <ref name="pattern2"/>
    </choice>
  .../...
  <define name="pattern1">
    <element name="foo">
      <empty/>
    </element>
  </define>
  <define name="pattern2">
    <element name="foo">
      <empty/>
    </element>
  </define>]]>
        </programlisting>
        <para>or:</para>
        <programlisting>
<![CDATA[ pattern1|pattern2
 .../...
 pattern1=element foo{empty}
 pattern2=element foo{empty}]]>
        </programlisting>
        <para>This example is ambiguous because when we find an empty element <literal>foo</literal> we can't tell if it's been validated through <literal>pattern1</literal> or <literal>pattern2</literal> and it's an ambiguous hedge grammar (rather than an ambiguous regular expression) because the ambiguity is spread over two hedges, i.e. two definitions of the element <literal>foo</literal>.</para>
        <para>Again, it has been demonstrated that ambiguous regular hedge grammars can be rewritten in unambiguous forms but the disambiguation must be done at the level of the grammar itself and does often requires heavy changes to the structure of the schema.</para>
        <para>The exercise of disambiguating regular hedge grammars can get significantly complicated when compositions of named patterns and grammars are involved. For instance, maintaining non ambiguous patterns while combining definitions by choice means that you need to exclude all the instance nodesets valid per the original definition from the pattern given as a choice and this isn't always possible without modifying the included schema. Consider for instance this pattern:</para>
        <programlisting>
<![CDATA[ <define name="namedPattern">
  <ref name="first"/>
 </define>]]>
        </programlisting>
        <para>or:</para>
        <programlisting>
<![CDATA[ namedPattern=first]]>
        </programlisting>
        <para>If we need to add an optional <literal>second</literal> pattern it may seem natural to combine it by choice as:</para>
        <programlisting>
<![CDATA[ <define name="namedPattern" combine="choice">
  <ref name="first"/>
  <optional>
   <ref name="second"/>
  </optional>
 </define>]]>
        </programlisting>
        <para>or:</para>
        <programlisting>
<![CDATA[ namedPattern|=first,second?]]>
        </programlisting>
        <para>The result of the combination is equivalent to:</para>
        <programlisting>
<![CDATA[ <define name="namedPattern">
  <choice>
   <ref name="first"/>
   <group>
    <ref name="first"/>
    <optional>
     <ref name="second"/>
    </optional>
   </group>
  </choice>
 </define>]]>
        </programlisting>
        <para>or:</para>
        <programlisting>
<![CDATA[ namedPattern=first|(first,second?)]]>
        </programlisting>
        <para>This gives us an ambiguous pattern. Of course, outside the context of a pattern combination, this would be trivial to rewrite as:</para>
        <programlisting>
<![CDATA[ <define name="namedPattern">
  <ref name="first"/>
  <optional>
   <ref name="second"/>
  </optional>
 </define>]]>
        </programlisting>
        <para>or:</para>
        <programlisting>
<![CDATA[ namedPattern=first,second?]]>
        </programlisting>
        <para>but in this case, we won't get there directly by pattern combination and we need to take the problem under a different angle and consider that we must leave in the alternative to the original definition only things which would are not already allowed. In other words, we need to remove from our target of &quot;the first pattern followed by an optional second pattern&quot; the case where the first pattern is not followed by the second one. The alternative will thus be between the first pattern alone and the first pattern followed by a second one:</para>
        <programlisting>
<![CDATA[ <define name="namedPattern" combine="choice">
  <choice>
   <ref name="first"/>
   <group>
    <ref name="first"/>
    <ref name="second"/>
   </group>
  </choice>
 </define>]]>
        </programlisting>
        <para>or:</para>
        <programlisting>
<![CDATA[ namedPattern=first|(first,second)]]>
        </programlisting>
        <para>With this target in mind, we can rewrite our combination as:</para>
        <programlisting>
<![CDATA[ <define name="namedPattern" combine="choice">
  <ref name="first"/>
  <ref name="second"/>
 </define>]]>
        </programlisting>
        <para>or:</para>
        <programlisting>
<![CDATA[ namedPattern|=first,second]]>
        </programlisting>
        <para>If we want to avoid ambiguous hedge grammars, we also need to be careful when combining named patterns by choice: without knowing how pattern1 and pattern2 are defined, it's just impossible to say if:</para>
        <programlisting>
<![CDATA[ <choice>
  <ref name="pattern1"/>
  <ref name="pattern2"/>
 </choice>]]>
        </programlisting>
        <para>or:</para>
        <programlisting>
<![CDATA[ pattern1|pattern2]]>
        </programlisting>
        <para>is ambiguous or not.</para>
      </sect3>
      <sect3>
        <title>Name class ambiguity</title>
        <para>Another source of ambiguity is when name classes used in different alternatives of a choice overlap. Again, a schematic example of such overlap would be:</para>
        <programlisting>
<![CDATA[      <choice>
        <element name="foo">
          <empty/>
        </element>
        <element>
          <anyName/>
          <empty/>
        </element>
      </choice>]]>
        </programlisting>
        <para>or:</para>
        <programlisting>
<![CDATA[ element foo{empty} | element * {empty}]]>
        </programlisting>
        <para>This is ambiguous since the name class <literal>anyName</literal> includes the name class matching only the name <literal>foo</literal> and an element <literal>foo</literal> would be valid per both branches of the choice pattern.</para>
        <para>The <literal>except</literal> name class does save our lives for name class ambiguity since it lets us remove the overlap from one of the alternatives and this pattern can easily be rewritten as a non ambiguous choice pattern:</para>
        <programlisting>
<![CDATA[      <choice>
        <element name="foo">
          <empty/>
        </element>
        <element>
          <anyName>
            <except>
              <name>foo</name>
            </except>
          </anyName>
          <empty/>
        </element>
      </choice>]]>
        </programlisting>
        <para>or:</para>
        <programlisting>
<![CDATA[ element foo{empty} | element * - foo {empty}]]>
        </programlisting>
        <para>Or more simply:</para>
        <programlisting>
<![CDATA[        <element>
          <anyName>
            <except>
              <name>foo</name>
            </except>
          </anyName>
          <empty/>
        </element>]]>
        </programlisting>
        <para>or:</para>
        <programlisting>
<![CDATA[ element * {empty}]]>
        </programlisting>
        <para>Note that the fact that name classes overlap is not enough to make an ambiguous pattern. For instance:</para>
        <programlisting>
<![CDATA[      <choice>
        <element name="foo">
          <attribute name="bar">
            <empty/>
          </attribute>
        </element>
        <element>
          <anyName/>
          <empty/>
        </element>
      </choice>]]>
        </programlisting>
        <para>or:
	element foo{attribute bar{empty}} |
	element * {empty}}</para>
        <para>is no ambiguous since the content model of the elements with the two name class do not overlap. Making our <literal>bar</literal> attribute optional:</para>
        <programlisting>
<![CDATA[      <choice>
        <element name="foo">
          <optional>
            <attribute name="bar">
              <empty/>
            </attribute>
          </optional>
        </element>
        <element>
          <anyName/>
          <empty/>
        </element>
      </choice>]]>
        </programlisting>
        <para>or:
	element foo{attribute bar{empty}?} |
	element * {empty}}</para>
        <para>is enough to make our pattern ambiguous. However, this pattern is strictly equivalent to the preceding one which means that we know how to rewrite it in a non ambiguous way.</para>
        <para>Finally, note that name class ambiguity may be considered as an extension to regular hedge grammar ambiguity. When we have been writing:</para>
        <programlisting>
<![CDATA[  element foo{empty} | element foo{empty}]]>
        </programlisting>
        <para>which after simplification is an example of regular hedge grammar ambiguity, the ambiguity comes from the fact that the name classes for both alternatives are the single value <literal>foo</literal> and thus do overlap.</para>
      </sect3>
      <sect3>
        <title>Ambiguous datatypes</title>
        <para>Datatype ambiguity is the one which is the most difficult to handle with Relax NG and that doesn't come from Relax NG itself but rather from the fact that datatype libraries are not built-in and are kind of opaque and less flexible than other patterns or name classes.</para>
        <para>A basic example of ambiguous datatypes is:</para>
        <programlisting>
<![CDATA[    <element name="foo">
      <choice>
        <data type="boolean"/>
        <data type="integer"/>
      </choice>
    </element>]]>
        </programlisting>
        <para>or:</para>
        <programlisting>
<![CDATA[ element foo{xsd:boolean|xsd:integer}]]>
        </programlisting>
        <para>Since the lexical space of the two possible datatypes do overlap (0 and 1 are valid W3C XML Schema boolean and integers), there is no way to determine what is the datatype an element foo with a value <literal>0</literal> or <literal>1</literal>. We have no except pattern available to remove the lexical space of a datatype from the lexical space of another datatype and, the only way to disambiguate such pattern is using parameters when the datatype library provides any. In the case of W3C XML Schema, the parameter to use if we want to work on the lexical space is the pattern parameter and we could remove <literal>0</literal> and <literal>1</literal> from the lexical space of the boolean datatype by either specifying the lexical space of boolean as being explicitly <literal>true</literal> or <literal>false</literal>:</para>
        <programlisting>
<![CDATA[    <element name="foo">
      <choice>
        <data type="boolean">
          <param name="pattern">true|false</param>
        </data>
        <data type="integer"/>
      </choice>
    </element>]]>
        </programlisting>
        <para>or:</para>
        <programlisting>
<![CDATA[ element foo{
   xsd:boolean{pattern="true|false"}
   |xsd:integer
 }]]>
        </programlisting>
        <para>Or by removing the lexical space of integer from boolean:</para>
        <programlisting>
<![CDATA[    <element name="foo">
      <choice>
        <data type="boolean">
          <param name="pattern">[^0-9]*</param>
        </data>
        <data type="integer"/>
      </choice>
    </element>]]>
        </programlisting>
        <para>or:</para>
        <programlisting>
<![CDATA[ element foo{
  xsd:boolean{pattern="[^0-9]*"}
  |xsd:integer
 }]]>
        </programlisting>
        <para>That's not much fun for more complex cases, but that's the only hope we have to disambiguate such ambiguities.</para>
      </sect3>
    </sect2>
  </sect1>
  <sect1>
    <title>The downsides of ambiguous and non deterministic content models</title>
    <para>Again, if you're only interested in using a Relax NG schema for validation which, after all, is the primary goal of Relax NG, it is perfectly fine to design and use non deterministic and even ambiguous schemas. The downsides of ambiguous schemas appear when we want to use Relax NG schemas for adding validation information to the instance documents or use a Relax NG schema for guided editing and the downsides of non deterministic schemas only appear when we want to be able to translate our schemas into a W3C XML Schema.</para>
    <sect2>
      <title>Instance annotations</title>
      <para>What I'll be calling instance annotation in this book is the ability to attach to the instance document information gathered during the validation to facilitate its processing. Instance annotation is probably one of the most promising paths to automating XML document processing and its applications cover domains such as datatype assignment (which is one of the basis of XQuery 1.0, XPath 2.0 and XSLT 2.0), data binding (probably the only way to automate the creation of objects from XML documents and the creation of XML documents from objects) and XML guided editing.</para>
      <para>Some tools may have more stringent requirements depending on their algorithms (for instance, a SAX based streaming tool might want to impose deterministic schemas), but in theory (and in general), it is sufficient for the applications of instance annotations to insure that the annotations are consistent and this can be achieved if the schema is unambiguous.</para>
      <para>Note that even this condition isn't always required and that these requirements are application dependents. Consider for instance a databinding application which needs to know the content model of each element. This application might be in trouble to determine which content model to use if it finds a pattern such as:</para>
      <programlisting>
<![CDATA[ element foo {first?,second}
 |element foo {second,third?}]]>
      </programlisting>
      <programlisting>
<![CDATA[
 first=element first{xsd:integer}
 second=element second{xsd:token}
 third=element third{xsd:boolean}]]>
      </programlisting>
      <para>and an element foo with a content pattern matching the <literal>second</literal> pattern. Should it bind it into an object allowing an optional <literal>first</literal> or into an object allowing an optional <literal>third</literal>? Such ambiguity is likely to be an issue for this application. On the other hand, if all you need is to perform simple type assignment, this schema is perfectly fine since even though it is ambiguous, there is no ambiguity on datatype assignment.</para>
      <para>As a bottom line, we can say that chasing ambiguity in your Relax NG schemas may be considered a good practice if you have in mind instance annotation applications at large, you must also check the tools which you will be using since they can have either more stringent or more relaxed requirements.</para>
    </sect2>
    <sect2>
      <title>Compatibility with W3C XML Schema</title>
      <para>I have promised to give an example of unambiguous patterns which is not deterministic and can't be rewritten in a deterministic form and here it is! Let's consider a pattern describing a book as a sequence of odd and even pages:</para>
      <programlisting>
<![CDATA[      <zeroOrMore>
        <ref name="odd"/>
        <ref name="even"/>
      </zeroOrMore>
      <optional>
        <ref name="odd"/>
      </optional>]]>
      </programlisting>
      <para>or:</para>
      <programlisting>
<![CDATA[ (odd, even)*, odd?]]>
      </programlisting>
      <para>This pattern is not ambiguous since for any valid combinations of odd and even pages it is possible to know which pattern has matched each of the pages. It can't be deterministic since for each odd page, you need to look ahead at the next one to see if it is the last before knowing if an even page is required in next position.</para>
      <para>W3C XML Schema requires deterministic content models under the name of &quot;Unique Particle Attribution&quot; and &quot;Consistent Declaration&quot; rules and just can't describe such a simple and useful content model!</para>
      <para>Another example of non deterministic pattern is:</para>
      <programlisting>
<![CDATA[ <choice>
  <element name="foo">
   <attribute name="bar"/>
  </element>
  <element name="foo">
   <element name="bar">
    <text/>
   </element>
  </element>
 </choice>]]>
      </programlisting>
      <para>or:</para>
      <programlisting>
<![CDATA[ element foo {attribute bar} | element foo {element bar {text}}]]>
      </programlisting>
      <para>This one would seem easier to translate. At least, it can be factorized and rewritten as a deterministic pattern in Relax NG as:</para>
      <programlisting>
<![CDATA[ <element name="foo">
  <choice>
   <attribute name="bar"/>
   <element name="bar">
    <text/>
   </element>
  </choice>
 </element>]]>
      </programlisting>
      <para>or:</para>
      <programlisting>
<![CDATA[ element foo {attribute bar| element bar {text}}]]>
      </programlisting>
      <para>Unfortunately, this doesn't help to translate our schema into W3C XML Schema since this language doesn't know how to handle this type of situations mixing constraints on sub elements and attributes without using dark hacks with key definitions which don't work in all the cases.</para>
      <para>Making sure that your schemas are deterministic is thus a good practice when you plan to translate your schemas into W3C XML Schema schemas but unfortunately not a guarantee that they will translate gracefully. The only rule I can give if you want to make sure that your schemas will be easy to translate is to check the result of translation frequently as you write your schema and hope that James Clark will continue to improve the conversion algorithm!</para>
      <para>On the other hand, note that W3C XML Schema deals nicely with datatype ambiguities. If we take or example of datatype ambiguity:</para>
      <programlisting>
<![CDATA[ element foo{xsd:boolean|xsd:integer}]]>
      </programlisting>
      <para>you will be surprised to know that it translates gracefully into:</para>
      <programlisting>
<![CDATA[  <xs:element name="foo">
    <xs:simpleType>
      <xs:union memberTypes="xs:boolean xs:integer"/>
    </xs:simpleType>
  </xs:element>]]>
      </programlisting>
      <para>and that this is not considered as ambiguous by W3C XML Schema which has added a rule to say that when several datatypes where grouped &quot;by union&quot; which is basically what our choice between datatype does, a processor should stop after the first type which matches and not evaluate the next alternatives.</para>
    </sect2>
  </sect1>
  <sect1>
    <title>Some ideas to make disambiguation easier</title>
    <para>To close this chapter I'd like to present some ideas which would facilitate our lives in disambiguating schemas.</para>
    <sect2>
      <title>Generalized <literal>except</literal> pattern</title>
      <para>In the different forms of ambiguity, name classes has been the easiest one to disambiguate. Why is this? Not because name classes are inherently simpler than regular expressions or datatypes: all of them are about defining sets of things that can happen in a XML documents and I would argue that they are very similar. The reason why name classes have been easier to disambiguate is because they have a first class <literal>except</literal> operator and if we had the same level of support for patterns and datatypes, we could more easily disambiguate them.</para>
      <para>Applied to datatypes, this is already possible to some extend and away to disambiguate our example:</para>
      <programlisting>
<![CDATA[ element foo{xsd:boolean|xsd:integer}]]>
      </programlisting>
      <para>is to write:</para>
      <programlisting>
<![CDATA[ element foo{ (xsd:boolean - xsd:integer) |xsd:integer}]]>
      </programlisting>
      <para>A value which is only integer will obviously match only the right alternative. A value which is only boolean (i.e. <literal>true</literal> and <literal>false</literal>) will match the left alternative and a value which is both a boolean and an integer (i.e. <literal>0</literal> and <literal>1</literal>) will match the first condition of the left alternative (<literal>xsd:boolean</literal>) but will not match the exception clause.</para>
      <para>Unfortunately, this can't be generalized out of the scope of <literal>data</literal> patterns (note that the examples given below with the except (<literal>-</literal>) operator are not valid Relax NG).</para>
      <para>If this could be generalized, applied to an ambiguous regular expression such as:</para>
      <programlisting>
<![CDATA[ two|(one?,two+,three*)]]>
      </programlisting>
      <para>We would be able to write:</para>
      <programlisting>
<![CDATA[ two|((one?,two+,three*)-two)]]>
      </programlisting>
      <para>Of course, this can be developed and rewritten with the existing Relax NG patterns, but that would give a new level of flexibility to the language.</para>
    </sect2>
    <sect2>
      <title>Explicit disambiguation rules</title>
      <para>The second idea I'd like to give is far less disruptive and is just the realization that these ambiguities are just ambiguous because we have not decided anything to rule them out. There are plenty of examples in other computer languages of ambiguities which have been partially or fully ruled out such as for XSLT templates, order of evaluation of statements in programming languages or as we've seen in the section about W3C XML Schema union of datatypes.</para>
      <para>There is absolutely nothing preventing writing a specification defining a priority for the alternatives to be used by applications interested in instance annotation at large when they encounter ambiguities.</para>
      <para>This specification wouldn't need to apply to Relax NG processors interested only in validation and would not compromise their optimizations. It would only apply to NG processors performing instance annotation and guarantee a consistent and interoperable type annotation for schema which are today considered as ambiguous.</para>
      <para>The rule could be as simple as &quot;use the first alternative in document order&quot; or it could also take into account additional factors such as giving a lesser precedence to included grammars like XSLT does with stylesheet imports.</para>
    </sect2>
    <sect2>
      <title>Accepting ambiguity</title>
      <para>The third idea has been proposed by Jeni Tennison on the xml-dev mailing list: instead of trying to fight against ambiguity, why not accept it? Why couldn't we acknowledge that something can have several datatypes (or model) and have at the same type a datatype &quot;A&quot; and &quot;B&quot;? Why a value couldn't be at the same type an <literal>integer</literal> and a <literal>boolean</literal>?</para>
      <para>This idea would have a serious impact on specifications such as XPath 2.0 which assign a single datatype to each simple type element and attribute, but that would be much more compatible with the principle of markup which is only the projection of a structure over a document. It often happen that a piece of text may have several meaning, acknowledging that elements and attributes may have belong to datatypes at the same time just seems something obvious to do.</para>
    </sect2>
  </sect1>
</chapter>