Motivation

Currently, Apache Ignite is being shipped as a single monolith binary with lots of excess modules and libs, the majority of which are not required in most use case scenarios.
Also, current upgrade process is too tied to binary delivery, requiring users to copy the working directory, additional libs, and other configurations between version upgrades, which is rather error-prone.

This IEP targets to solve both problems the following way:

1. Enhance ignite.sh & other control scripts to untie it from IGNITE_HOME and use Maven as the main additional lib management system
2. Update command-line utility to provide the minimal required module management
3. Provide command-line extensibility which is in line with the new distribution model

Final target – recreate binary delivery in a way, that will allow a user to upgrade Apache Ignite instance with only downloading a new binary and executing the same ignite command for the new version.

Description

Binary Delivery Revamp

1. The binary delivery is narrowed down to a single ignite command-line utility which is the essential toolchain for installing ignite modules, managing classpath, and node lifecycle.

2. Correspondingly update RPB/DEB packages to include the ignite CLI utility

Additional Libs Management System

Replace the shipping of all Ignite libs with Maven-powered dependency management:

1. The utility is equipped with a set of commands to install arbitrary Maven artifacts using Maven coordinates (e.g. mvn:org.apache.ignite:ignite-core:2.9.0)
2. The utility has a set of built-in aliases for Maven artifacts to simplify default modules installation (e.g. server->[mvn:org.apache.ignite:ignite-core:2.9.0, mvn:org.apache.ignite:ignite-indexing:2.9.0, mvn:org.apache.ignite:ignite-spring:2.9.0]
3. The utility should be able to support additional Maven repositories, as well as external resource locations (SFTP, HTTP(s), etc)

Configurations Enhancements

For well-defined and predictable library management, the utility has a number of indirections when resolving work, configuration, and library storage paths. The resolution path depends on whether ignite was configured locally or globally for a machine/container.

Versioning & Installation

Ignite installation from a user perspective narrows down to a single package containing the command-line tool install via one of the package managers (apt-get, rpm, homebrew) or a manual download. Once the command-line tool is installed, further modules management is performed from the tool.

The utility version directly matches the server and client library versions which this utility will manage, i.e. ignite v.${version} will use org.apache.ignite:ignite-core:${version}, org.apache.ignite:ignite-spring.${version} and so on.

In order to update Ignite, a user updates the single ignite package, which will later take care of updating installed modules. A user can easily switch between ignite versions using regular update-alternatives (or similar) tools.

Library and work dir resolution

To differentiate between local and global configuration mode, the utility first attempts to read the .ignitecfg file in the user local directory ${user.home}. If this file exists, then the library and work directory paths are defined in there. If the file does not exist, the utility attempts to read the .ignitecfg file in a predefined system folder (e.g. it may be /etc/ignite for Linux systems, /Library/Ignite for MacOS, Program Files\Ignite for Windows). If this file exists, then the library and work directory paths are defined there. If none of the files exist, then the library and work directories default to ${user.home}/ignite-bin and ${user.home}/ignite-work.

file = if ${user.home}/.ignitecfg exists
           ${user.home}/.igniterc
       else if ${system.default}/.ignitecfg exists
           ${system.default}/.ignitecfg
       else
           None

if file is None {
    libsDir = ${user.home}/ignite-bin
    workDir = ${user.home}/ignite-work
} else {
    libsDir, workDir = read from file
}

Modules management

Modules and external libraries can be installed and uninstalled using ignite install or ignite uninstall commands. The command accepts a built-in alias for one or many Maven coordinates, maven coordinates, or a URI for external resource. The set of installed modules is written to a config file ignite.modules located in the ${workDir} folder, resolved previously (the configuration file is updated whenever a module is installed or uninstalled). Upon module installation, ignite resolves corresponding artifacts and its dependencies from Maven and places them to the ${libsDir}/${version} folder. Additionally, ignite pre-builds and saves a classpath formed by all installed modules in a separate file for faster node start-up.

A default directory structure of installing Ignite 3.0.1 and then upgrading to Ignite 3.0.2 will look as follows:

/
|
+---+Library/Ignite                 ignite
|    |                                 |
|    +----3.0.1                        |
|    |    |                            |
|    |    +----ignite-3.0.1            |
|    |                                 |
|    +----3.0.2                        |
|         |                            |
|         +----ignite-3.0.2 <----------+
|
+----Users/John
     |
     +----ignite-bin
     |    |
     |    +----3.0.1
     |    |    |
     |    |    +----ignite-core-3.0.1.jar
     |    |    |
     |    |    +----plugin-cli-0.0.1.jar
     |    |
     |    +---+3.0.2
     |         |
     |         +----ignite-core-3.0.2.jar
     |         |
     |         +----plugin-cli-0.0.1.jar
     |
     +----ignite-work
          |
          +----ignite.modules={ignite-core,plugin-cli:0.0.1}
          |
          +----ignite.classpath={ignite-core-3.0.2.jar:plugin-cli-0.0.1.jar}

Running Ignite

Unlike 2.x ignite.sh script, the ignite run command will start Ignite node in a separate process without attaching it to the current terminal. This is done to simplify the node management and support interactive REPL mode for ignite. If no modules are installed (the default set is missing), the run command will report an error asking the user to install the modules first (this is done intentionally to prevent accidental downloads and internet access attempts on production servers).

Starting a new Ignite node may require some local initialization to have an ability to set up node-local properties and create local metastorage. The initialization will be performed via a local command and will take the node consistent ID. Similarly, the ignite run command will require the node consistent ID. If the node metastorage is not initialized, the run command will initialize it with defaults and run the node; if it is initialized, the node will be started with existing metastorage.

Cluster and local commands, management endpoint

Commands in the utility can be split into two groups: local commands (such as ignite run, ignite install), which can be executed without connection to a running cluster, and cluster commands (such as ignite baseline), which require connection to one of the ignite nodes. ignite cluster commands must have a straightforward and unambiguous mapping to the cluster management HTTP REST endpoints. With such a strict mapping we can achieve:

• easy implementation of all cluster commands at once which will solely require a list of available endpoints and arguments format
• command and argument autocompletion with possible autocompletion for dynamic arguments (for example, auto-complete node consistent ID for the ignite baseline command)
• the ability for a user to programmatically manage Ignite cluster from any platform/tool supporting HTTP (e.g. curl)

Pluggable commands

The ignite utility should be extensible and allow plugging additional commands (both cluster and local). Upon the utility startup, it will scan the ${libsDir}/${version} folder for files with -cli suffix and search such files for command definitions. This allows a user to install additional commands using the same ignite install command. 

Risks and Assumptions

After installing ignite, a user will still need one-time access to the internet to download the rest of the libraries. This, however, carries low risk as the initial setup can be performed either via curl <link_to_ignite> && ./ignite install command or a custom packed .sh script which will execute the install command after download.

Discussion Links

Development list: IEP-52: Binary Delivery & Upgradability Enhancements

Reference Links

n/a

Tickets

n/a