To be Reviewed By: October 26, 2020

Authors: Dale Emery

Status: Draft | Discussion | Active | Dropped | Superseded

Superseded by: N/A

Related: N/A

Problem

Numerous Geode classes resolve relative file paths against the JVM’s working directory—the value of the user.dir  system property. Some classes do this explicitly, by resolving paths against the value of user.dir . Other classes refer to the JVM’s working directory indirectly, either by using relative paths or by resolving paths against “.”  or “” .

Resolving files against the JVM’s working directory makes it impossible to test many Geode features in parallel outside of a container.

Every Gradle test worker JVM for a given task type (e.g. distributedTest) has the same working directory. When tests run in parallel outside of a container, if multiple tests write to the same file path relative to the JVM’s working directory, those tests all write to the same file. The tests interfere with each other's files, causing swarms of failures.

Anti-Goals

None identified.

Solution

Make Geode resolve relative paths against a configurable path defined by a new geode.working-dir  system property.

Instead of accessing the property directly, Geode code will get the working directory from a new WorkingDirectory  class, by calling static methods that return the working directory as a java.nio.file.Path  or a java.io.File. To maintain Geode’s current behavior, these methods return the value of user.dir  if geode.working-dir  is not set.

Optionally (TBD): These methods can ensure that the returned path is absolute and normalized, and that it refers to an existing directory.

Changes and Additions to Public Interfaces

This proposal adds a new public system property: geode.working-dir .

Any documentation that refers to user.dir will need to refer instead to geode.working-dir .

Any assumptions made by shell scripts that ship with Geode will have to account for this property.

Performance Impact

I expect that the additional cost to resolve paths to a canonical path will be minimal.

Backwards Compatibility and Upgrade Path

By default, if the new geode.working-dir  property is not set, Geode will behave as it currently does. Users who do not set this property will experience no change.

If the user specifies a geode.working-dir other than the JVM's working directory, the effect on backward compatibility, the rolling upgrade process, and any shell scripts provided by either Geode or the user is TBD.

Prior Art

None known.

FAQ

None yet.

Errata

None yet.

  • No labels

7 Comments

  1. Anilkumar Gingade

    Thanks for taking this up. It helps user to configure product in their required environment (considering the security requirements within the org).

    Few questions:

    In the absence of Disk Dir configuration, the persistence data is written into working dir; with this change? 

    All the product related configuration files (like geode.properties, geode-security.properties) will they be read from the new user dir?  

    During rolling upgrade or restart (currently it says TBA); can we assume that if the users sets this property, they are responsible to copy all the related files to the newly set working dir?

    1. Dale Emery

      For Disk Dir configuration… There are several configuration properties (like the default Disk Dir) that specify directories. For each of those, I propose:

      • If the property specifies an absolute path, use it.
      • If the property specifies a relative path, resolve it against the value of geode.working-dir.
      • If the property is not specified, it defaults to the value of geode.working-dir.

      For product configuration files… The product will search for its configuration files in the specified geode.working-dir. Otherwise, different tests that run in parallel will not be able to specify different configuration files. Perhaps the product could search geode.working-dir first, and then search the JVM's working dir, but I'm hoping this will be unnecessary.

      For rolling upgrades and restarts… If the user does not specify a geode.working-dir, the product will work as it does now. If they do specify a geode.working-dir, I don't know enough about these scenarios to make a recommendation. I would appreciate comments, ideas, or concerns about that.

  2. Jinmei Liao

    I like the idea of using system property to specify a working dir, all properties files for startup the process will be read from that dir first. There is one more possible conflict to consider: currently serverLauncher and locatorLauncher take a working dir as a parameter, what if it conflicts with this system property, who takes precedence?

    1. Dale Emery

      Interesting question. I'm of two minds:

      1. Because the launchers can be invoked on the command line, launcher properties should win.
      2. Because the launchers can be invoked programmatically, system properties should win.

      Current use of the launchers' working dir properties offers more puzzles than guidance:

      • ServerLauncher's working dir wins… but as far as I can tell it is used only for the "controllable process" files (e.g. the pid file).
      • LocatorLauncher's working dir wins… but if the command is "start," the launcher insists that the working dir must exactly match the JVM's working dir.

      Are there other existing launcher properties that can conflict with system properties in this way? If so, they may provide guidance.

  3. Bill Burcham

    I see the need to solve this problem, but I don't think we should use a Java system property to do it.

    A system property is a global variable. We have found that global variables confound testing. (If I want to run tests concurrently within a JVM then system properties present an insoluble problem.) We need to stop adding them to Geode.

    Rather than specifying a system property, let's dig in to the line of questioning above. I propose we identify the Geode entry points, and for each entry point, specify a way to set the Geode working directory. As already pointed out, we have a precedent:

    1. LocatorLauncher
      1. LocatorLauncher.Builder.setWorkingDirectory(Path)
      2. main(String[]): <any-command> --dir=<pathspec>
    2. ServerLauncher
      1. ServerLauncher.Builder.setWorkingDirectory(Path)
      2. main(String[]): <any-command> --dir=<pathspec>
    3. (…are there others?)

    The game is to identify any other entry-points and to lock down the semantics for each (and semantic changes too.)

    The necessary changes that have already been identified:

    • expand scope of working dir specified via ServerLauncher beyond "controllable process" files
    • relax requirement in LocatorLauncher, that working dir match JVM's working dir
    • (changes to semantics of various other path specifications e.g. disk dir: absolute paths are respected but relative paths are evaluated relative to Geode working directory)

    1. Dale Emery

      I am currently analyzing Geode's references to the JVM's working directory. For some of those, there's readily available object from which the code could obtain the working directory (a config, a cache that has a config, a region associated with a cache that has a config, …). For many uses, I don't yet see a good non-singleton source. I'm still looking.

  4. Dale Emery

    Here are the ways in which Geode references the JVM's working directory, either explicitly or implicitly: