Page History

(As-is-built 2021-0910-2806)

This page describes an enhancement to the DFDL language, the Daffodil DFDL Layering extension, enabling DFDL schemas using this extension to describe data formats in such a way as to enable algorithmic computations of CRCs, Checksums, Parity bits, etc. These are checked when parsing, to insure data validity and/or well-formedness, and are recomputed and incorporated into the data infoset when unparsing data. 

Principles of Operation

Daffodil's layers are able to read and write DFDL variables. Variables are used to provide parameters to layer computations, and to receive results back from layer computations. 

...

• Layer transformations are able to read and write DFDL variables, which means they need access to the parse or unparse state of the computation.
• When parsing, the DFDL variables read or written by a layer transformation must be defined or it is a runtime an SDE. 
• When unparsing, DFDL variables read by a layer transformation must either have a value, or the read must cause causes suspension of the layer transformation if the variable is not yet defined. This layer transformation must be is resumed once the variable gets its value. 
• When unparsing, computations of dfdl:outputValueCalc, or dfdl:length or other runtime-valued expressions can reference DFDL variables that will are (to be) set by a layer transform. Such computations must will suspend if the layer transformation has not yet set the variable, and resume when the variable is set. 

Examples

Worked examples showing layering transformations that implement checksums and parity are:

• IPv4 packets - See the daffodil-test src/test/resources/org/apache/daffodil/layers/IPv4.tdml test, related IPv4.dfdl.xsd schema. This is fake, but very much like IPv4. 
  
  • There is also a PR to update the ethernetIP github DFDL schema project to compute checksums.  https://github.com/DFDLSchemas/ethernetIP/pull/1. However, there are checksums in the IPv4Header, UDP, TCP, ICMP, etc. So the work involved here is extensive.
  • This work likely should be done only after

    Jira
    server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key DAFFODIL-1927

    is complete so that the pluggable layers that compute these checksums can be created as a plugin. 
• GPS data with parity - See the GPS-SPS DFDL schema on the DFDL schemas github site.
  • TBD: currently a pull request: https://github.com/DFDLSchemas/gps-sps/pull/1
  • This work likely should be done only after

    Jira
    server ASF JIRA serverId 5aa69414-a9e9-3523-82ec-879b028fb15b key DAFFODIL-1927

    is complete so that the pluggable layers for this parity can be done as a plugin. 
• Check-digit computation - See the daffodil-test src/test/resources/org/apache/daffodil/layers/TestCheckDigit.tdml test file and related checkDigit.dfdl.xsd schema. 

...