Enable separate namespace for adapter schema in FDML

[dlindhol]

We currently have adapter specific schema that "redefine" the core schema, extending it with additional properties needed by the Adapter code. FDML files then have a single default namespace via something like:

xsi:noNamespaceSchemaLocation="http://latis-data.io/schemas/1.0/fdml-with-text-adapter.xsd"

The goal with this issue is to have the default schema be the core fdml schema (http://latis-data.io/schemas/1.0/fdml.xsd) with separate named schema for the adapter with something like:

xmlns:a="http://latis-data.io/schemas/1.0/text-adapter.xsd"

All adapter specific elements/attributes in the fdml would then be prefixed with the namespace (e.g. "a:delimiter").

Ultimately, for another ticket, the FdmlReader will package all the information in the adapter's namespace into an adapter config.

[dlindhol]

Note that our "redefine" approach allows us to specifically add valid properties to core structures. This new approach may require us to relax the core schema to allow any additional elements/attributes with a different namespace, if we don't already.

[lindholc]

With a few changes to the base FDML schema:

@@ -1,12 +1,14 @@
 <?xml version="1.0" encoding="UTF-8"?>
-<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
-           version="1.0">
+<xs:schema xmlns="http://latis-data.io/schemas/2.0/fdml"
+           xmlns:xs="http://www.w3.org/2001/XMLSchema"
+           targetNamespace="http://latis-data.io/schemas/2.0/fdml"
+           version="2.0">
 
     <xs:element name="dataset">
         <xs:complexType>
             <xs:sequence>
                 <xs:element ref="source"/>
-                <xs:element ref="adapter"/>
+                <xs:element ref="adapterBase"/>
                 <xs:element ref="datatype"/>
             </xs:sequence>
             <xs:attribute name="id" type="xs:string" use="required"/>
@@ -22,15 +24,7 @@
         </xs:complexType>
     </xs:element>
 
-    <xs:element name="adapter">
-        <xs:complexType>
-            <xs:attributeGroup ref="adapter_attributes"/>
-        </xs:complexType>
-    </xs:element>
-
-    <xs:attributeGroup name="adapter_attributes">
-        <xs:attribute name="class" use="required"/>
-    </xs:attributeGroup>
+    <xs:element name="adapterBase" abstract="true"/>
 
     <xs:element name="datatype" abstract="true"/>
     <xs:element name="scalar" substitutionGroup="datatype">

And a new adapter schema:

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
           xmlns:fdml="http://latis-data.io/schemas/2.0/fdml"
           targetNamespace="http://latis-data.io/schemas/2.0/adapters/a">

  <xs:import namespace="http://latis-data.io/schemas/2.0/fdml"/>

  <xs:element name="adapter" substitutionGroup="fdml:adapterBase">
    <xs:complexType>
      <xs:attribute name="attr" use="required"/>
    </xs:complexType>
  </xs:element>

</xs:schema>

We can write FDML files like this:

<?xml version="1.0" encoding="UTF-8"?>

<dataset xmlns="http://latis-data.io/schemas/2.0/fdml"
         xmlns:a="http://latis-data.io/schemas/2.0/adapters/a"
         id="dataset">

  <source uri=""/>

  <!-- Note that this element is prefixed. -->
  <a:adapter attr="f"/>

  <function>
    <scalar id="time"
            units="MM dd yyyy"
            type="string"
            class="latis.time.Time"/>
    <scalar id="a" type="double"/>
  </function>
</dataset>

One thing this complicates is the class attribute. As written, every adapter schema would need to add the attribute.

[lindholc]

Making the class attribute required for all adapters was simple enough:

<!-- From fdml.xsd -->

<xs:element name="adapterBase" type="adapterBaseType" abstract="true"/>

<xs:complexType name="adapterBaseType">
  <xs:attribute name="class" use="required"/>
</xs:complexType>

<!-- From adapter-a.xsd -->

<xs:element name="adapter" substitutionGroup="fdml:adapterBase">
  <xs:complexType>
    <xs:complexContent>
      <xs:extension base="fdml:adapterBaseType">
        <xs:attribute name="attr" use="required"/>
      </xs:extension>
    </xs:complexContent>
  </xs:complexType>
</xs:element>

[lindholc]

One way to add attribute to scalars (and other things) is to name the scalar's type and make a new type extending the original, but that requires specifying the type of the scalar:

<!-- From fdml.xsd -->

<xs:element name="scalar" substitutionGroup="datatype" type="scalarBaseType"/>

<xs:complexType name="scalarBaseType">
  <xs:attributeGroup ref="scalar_attributes"/>
</xs:complexType>

<!-- From adapter-a.xsd -->

<xs:complexType name="adapterAScalarType">
  <xs:complexContent>
    <xs:extension base="fdml:scalarBaseType">
      <xs:attribute name="foo" use="required"/>
    </xs:extension>
  </xs:complexContent>
</xs:complexType>

<!-- From dataset.fdml -->
<scalar id="a" type="double" foo="" xsi:type="a:adapterAScalarType"/>

Note that the attribute we added (foo) doesn't need (and can't have) a prefix unless we set attributeFormDefault="qualified" in the adapter schema or set form="qualified" on all the attributes added by the adapter schema.

Ryan points out that nothing requires that a user use this new type, so an FDML file could validate but not work because things required by the adapter schema won't actually be used.

[lindholc]

Another way to add attributes to scalars is to allow any namespaced attributes and add global attributes to the adapter schema:

<!-- From fdml.xsd -->

<xs:attributeGroup name="scalar_attributes">
    <xs:attribute name="id" use="required"/>
    <!-- ... -->
    <xs:anyAttribute namespace="##other"/>
</xs:attributeGroup>

<!-- From adapter-a.xsd -->

<xs:attribute name="foo"/>

<!-- From dataset.fdml -->

<scalar id="a" type="double" a:foo=""/>

There are at least two major downsides to this approach:

1. We can't specify that any adapter attributes are required.
2. You can add whatever namespaced attributes you want, not just attributes added by adapters.

There's a way to limit the set of namespaces that attributes can be added from, but if we could enumerate the namespaces we wouldn't need to be doing this.