With the upcoming release of Daffodil 3.11.0 there are some unique challenges that stem from the change in Scala versions that has not been present in previous Daffodil releases.  Daffodil 3.11.0 targets Scala 2.13 and byte code for Scala 2.13 is not compatible with Scala 2.12, which all previous versions of Daffodil have targeted.  Below is a guide on how to transition a DFDL schema project from Scala 2.12 (Daffodil <= 3.10.0) to Scala 2.13 (Daffodil 3.11.0).  It will also include notes on how to maintain a DFDL schema project that targets both Scala 2.12 and 2.13.


Apache Daffodil SBT Plugin

While the use of the Apache Daffodil SBT Plugin (https://daffodil.apache.org/sbt/) is not necessarily required for developing a DFDL schema, its use is strongly encouraged and all following instructions will assume it is being used.  In order to target Daffodil 3.11.0 you will need Daffodil SBT Plugin version 1.4.0 or later.

Compatibility notes regarding Apache Daffodil SBT Plugin

While version 1.4.0 of the Apache Daffodil SBT Plugin targets Daffodil 3.11.0 by default, it can still be used to target older versions of Daffodil with one major caveat - Apache Daffodil SBT Plugin does not support targeting Scala 2.12 and 2.13 simultaneously.  For most DFDL schema projects this means having the following settings is not supported:

daffodilPackageBinVersions := Seq("3.10.0", "3.11.0")

This is because Daffodil 3.10.0 targets Scala 2.12 while Daffodil 3.11.0 targets Scala 2.13.  If you wish to build saved parsers for Daffodil 3.11.0 and older versions it will need to be done in multiple steps with small changes to the SBT settings.

For Daffodil 3.11.0

// No need to set daffodilVersion as version 1.4.0 of the Daffodil SBT plugin defaults to Daffodil 3.11.0

daffodilPackageBinVersions := Seq("3.11.0")

// From shell in project root directory

$ sbt packageDaffodilBin // Will produce a saved parser for Daffodil 3.11.0

For Daffodil 3.10.0 and earlier

daffodilVersion := "3.10.0" // Will also set the scalaVersion of the project to Scala 2.12

daffodilPackageBinVersions := Seq("3.10.0", "3.9.0", "3.8.0", ...)

// From shell in project root directory

$ sbt packageDaffodilBin // Will produce saved parsers for Daffodil 3.10.0, 3.9.0, 3.8.0, ...


The process for building Daffodil plugins or layers that are written in Scala is similar

For Daffodil 3.11.0

// No need to set daffodilVersion as version 1.4.0 of the Daffodil SBT plugin defaults to Daffodil 3.11.0

// From shell in project root directory
$ sbt publishLocal // This will publish a Scala 2.13 version of the plugin/layer

For Daffodil 3.10.0 and earlier

daffodilVersion := "3.10.0" // Will also set the scalaVersion of the project to Scala 2.12

// From shell in project root directory

$ sbt publishLocal // This will publish a Scala 2.12 version of the plugin/layer


Note that for both DFDL schema projects and Daffodil plugins/layers developed in Scala it may be necessary to make small changes to the Scala code to ensure that it compiles with both Scala 2.12 and 2.13.  This is usually pretty easy to accomplish.  If it is not possible it may be necessary to maintain separate branches targeting each version of Scala as it requires more than a couple setting changes to the SBT configuration.

Step by Step Guide

The following steps guide you through the necessary steps to target Daffodil 3.11.0 and therefore Scala 2.13 for a typical DFDL schema project

  1. Ensure you are using Apache Daffodil SBT Plugin v1.4.0 or later
  2. Ensure any required Daffodil plugins or layers are built and published targeting Daffodil 3.11.0
    1. Optionally build and publish targeting an older version of Daffodil in a separate step to also support older versions of Daffodil
  3. Ensure that any test code written in Scala for the DFDL schema project compiles with Scala 2.13
    1. Optionally ensure that the code also compiles with Scala 2.12 if also targeting older versions of Daffodi
  4. When creating saved parsers using sbt packageDaffodilBin perform the following to build saved parsers:
    1. $ sbt set daffodilPackageBinVersions := Seq("3.11.0") packageDaffodilBin // For Daffodil 3.11.0 this can ONLY contain "3.11.0"
    2. $ sbt set daffodilPackageBinVersions := Seq("3.10.0", "3.9.0", "3.8.0", ...) packageDaffodilBin // This CANNOT contain "3.11.0" when targeting older versions of Daffodil

Supporting both Daffodil 3.11.0 and earlier releases simultaneously with SBT subprojects

While the Daffodil SBT Plugin does not directly support targeting both Daffodil 3.11.0 and earlier versions simultaneously, it is possible to get this behavior by using SBT subprojects.  Below is an example of how to configure the build.sbt file for such a project:


lazy val commonSettings = Seq( // These settings are shared for both the default project (daf310) and the daf311 subproject.
  name := "dfdl-schema",

  organization := "org.apache",

  version := "1.0.0",


  libraryDependencies ++= Seq(
    "org.apache" % "some-linked-schema" % "1.0.0",
    // If we do not mark some-charset-plugin as a provided dependency, then this jar will have a
    // transitive dependency to one of the _2.12 or _2.13 some-charset-plugin jars, depending on
    // what version of Daffodil this project is built and published with. And this would mean
    // that any projects that depend on this this jar will also depend on a specific version of
    // the some-charset-plugin, which might not match the version of Daffodil they actually want to
    // use. By using provided, we avoid this issue, but it means projects depending on this this
    // project must manually add the appropriate some-charset-plugin dependency based on the
    // version of Daffodil they are using.
    "org.apache" %% "some-charset-plugin" % "1.0.0" % "provided",
    "org.apache.daffodil" %% "daffodil-sapi" % daffodilVersion.value % "test",
  ),


  daffodilPackageBinInfos := Seq(
    DaffodilBinInfo("/org/apache/some-schema.dfdl.xsd", Some("Message"),
      config = Some(sourceDirectory.value.getParentFile / "config.xml")   
  ),


  // Other project specific settings can go here
)


// This will be the default project as it is in the "." current working directory
lazy val daf310 = project.in(file("."))
  .enablePlugins(DaffodilPlugin)
  .settings(commonSettings)
  .settings(
    daffodilVersion := "3.10.0",
    daffodilPackageBinVersions := Seq("3.10.0")
  )
  .settings(
    // daf311 disables publishing since it creates duplicate jars. We still want to publish
    // "parser" artifacts if they are built, so we copy just those artifacts from daf311
    artifacts ++= (daf311/artifacts).value.filter(_.`type` == "parser"),
    packagedArtifacts ++= (daf311/packagedArtifacts).value.filter { case (art, _) => art.`type` == "parser" }
  )


lazy val daf311 = project.in(file("daf311"))
  .enablePlugins(DaffodilPlugin)
  .settings(commonSettings)
  .settings(
    daffodilVersion := "3.11.0",
    daffodilPackageBinVersions := Seq("3.11.0"),
  )
  .settings(
    // use the exact same sources as the root project
    sourceDirectory := baseDirectory.value.getParentFile / "src",
    // this is a schema jar so it is not cross versioned. Since it uses the exact same sources
    // as the root, it means the jars daf311 creates are exactly the same as the root jars. So
    // we disable publishing of these jars
    publish / skip := true
  )


  • No labels