...
To publish the R package on CRAN, there are a few steps we need to do first in order to ensure that binaries for Windows and macOS are available to CRAN. Jeroen Ooms <jeroenooms@gmail.com> maintains several projects that build C++ dependencies for R packages for macOS and Windows. We test copies of these same build scripts in our CI, and at release time, we need to send any changes we have and update the versions/hashes upstream.
https://github.com/r-windows/rtools-packages/pull/184 is an example of what needs to happen. It links to the other pull requests, and shows the workflow to follow. In sum, when When the release candidate is made, make make draft pull requests to each repository using the rc, updating the version and SHA, as well as any cmake build changes from the corresponding files in apache/arrow. (Each build script has a line that says "uncomment this to test with rc".) Jeroen may merge these PRs before the release vote passes, build the binary artifacts, and publish them in the right places so that we can do pre-submission checks (see below). After the release candidate vote passes, update these PRs to point to the official (non-rc) URL and mark them as ready for review. Jeroen will merge, build the binary artifacts, and publish them in the right places.
...
- The "autobrew" formula, a fork of Homebrew that is used in R package configure scripts to pull system dependencies on CRAN, where Homebrew is not available.
- Make a pull request to modify https://github.com/autobrew/homebrew-core/blob/master/Formula/apache-arrow.rb to update the version, SHA, and any changes to dependencies and build steps. Those dependency/build updates will have been already recorded in the copy we have of that formula in dev/tasks/homebrew-formulae/autobrew/apache-arrow.rb. This formula could be the same as the official Homebrew formula, but historically it has been a slimmer build of the C++ library (no Gandiva, for example). There are also some subtle differences because we need a static build.
- https://github.com/autobrew/homebrew-cran/blob/master/Formula/apache-arrow-static.rb, which is the same general deal–a Homebrew formula. autobrew/homebrew-core (the first one) is what is used to build the bottle that get downloaded on CRAN because they still use a macOS version that Homebrew no longer supports. This version builds the bottles for newer versions of macOS, which I think you'd only encounter if you did
install.packages("arrow", type = "source") on a (newer) mac. Our copy of this formula is at dev/tasks/homebrew-formulae/autobrew/apache-arrow-static.rb. - If there are new dependencies, you'll also need to update the script at https://github.com/autobrew/scripts/blob/master/apache-arrow to include any additions made to r/tools/autobrew. (These local copies of the autobrew scripts are tested in the nightly R package builds.)
- If you need to rebuild the bottles without changing the Arrow version (for example, you forgot to change a cmake flag and only realize it after Jeroen merges), be sure to add/bump the
revisionin the formula (like this) in order to get the bottles to be rebuilt.
- Update the analogous build scripts script for Windows. Adapt any build step changes in the new release from the ci/PKGBUILD file in apache/arrow, from which these scripts are based.
- rtools-packages, https://github.com/r-windows/rtools-packages/blob/master/mingw-w64-arrow/PKGBUILD. This is the rtools40 versionrtools-backports, https://github.com/r-windows/rtools-backports/blob/master/mingw-w64-arrow/PKGBUILD. This is the rtools35 versionfor R >= 4.0 only. As of 10.0.0, we use C++17, which is not supported on the rtools35 toolchain.
- PKGBUILD has a line that says "uncomment this to test with rc". For the draft PR, use the RC.
- After the release vote passes, either edit your PR (if it is still open) or open a new PR to switch back to the official release URL.
Once these binary prerequisites have been satisfied, we can submit to CRAN. Given the vagaries of the process, it is best if the R developers on the project verify the CRAN-worthiness of the package before submitting. Our CI systems give us some coverage for the things that CRAN checks, but there are a couple of final tests we should do to confirm that the release binaries will work and that everything runs on the same infrastructure that CRAN has, which is difficult/impossible to emulate fully on Travis or with Docker.
...