Log4j 2 2.x uses the PropertiesUtil class in log4j-api to access properties. It was enhanced in Log4j 2.10.0 to "normalize" property names. While this helped, with the growth in the number of properties Log4j uses managing them is getting harder.

We also desire to be able to have components enabled or disabled by default but to provide settings to override the default. With Log4j 2 3.x being much more modularized enablement of many things can be controlled simply by the presence of the jar that contains the relevant functionality. However, it is sometimes the case that we will want individual plugins to require specific enablement to be activated. The structure (or lack of) the current set of properties makes this difficult.

This proposal is to require all properties to follow a more well defined structure. The proposal for the structure of a Log4j 2 property is log4j2.{LoggerContextName}.{ComponentName}.{property}. To keep the documentation from becoming a mess the current properties in 2.x would NOT be supported in 3.0.

The LoggerContext has always been allowed to have a name. it currently is internally generated and used as the unique key to locate it based on its associated ClassLoader. This is unfortunate as there are cases where users would like to specify properties that apply only to a specific LoggerContext. This proposal would create a new private LoggerContext field named contextKey that is used in place of the contextName as the key in the LoggerContext Map. The various API layers would be modified to accept a contextName field when he LoggerContext is created. The context name will be immutable.

This will allow a configuration like the one shown below. log4j2 at the top level identifies the file as containing log4j properties. The element at the next level down is the name of the LoggerContext the enclosed properties apply to. A "*" as the LoggerContext name indicates the settings are the default values. Within that properties are grouped by the components they apply to. Note that the term "component" is used loosely here.

Note that JsonTemplateLayout includes a JsonReader. If that is moved into log4j-core we should be able to support JSON configuration and properties without any additional dependencies.  In addition, our Spring Boot support will allow the user to specify the JSON below in the application's bootstrap.yml file as an alternative to log4j2.component.properties and log4j2.component.yml.


{ "log4j2": {
    "My-App": {
      "JNDI": {
        "enableJMS": "true"
      }
      "Script": {
        "enableLanguages": [
	      "Groovy",
          "JavaScript"		
		]
      }
      "Configuration": {
        "mergeStrategy": "com.acme.log4j.CustomMergeStrategy",
        "location": "classpath:log4j2-My-App.xml",
        "statusLoggerLevel": "debug"
      }
   },       "*":  {
            "StatusLogger": {
                "defaultStatusLevel": "Info"
            }
       }}

Of course, the equivalent would also be available as properties, although the property names will change from their current values

log4j2.My-App.Jms.enableJndi=true
log4j2.My-App.Script.enableLanguages=Groovy,Javascript
log4j2.My-App.Configuration.mergeStrategy=com.acme.log4j.CustomMergeStrategy
log4j2.My-App.Configuration.location=classpath:log4j2-My-App.xml
log4j2.My-App.Configuration.statusLoggerLevel=debug
log4j2.*.StatusLogger.defaultStatusLevel=Info

Using this format allows restrictions to be placed on plugins:


@RequireProperty(component = "Lookup", property = "enableJndi", value = "true")
@Plugin(name = "jndi", category = StrLookup.CATEGORY)
public class JndiLookup extends AbstractLookup {}

This would prevent the Plugin from being loaded unless the requirement is met.

In the case of Scripting we would want the ScriptManager to be disabled unless one or more scripting language are enabled.  In the case of the ScriptManager it is loaded by the ScriptManagerFactory in Log4j 2 3.x, which is loaded via ServiceLoader. The ScriptManager then locates all the ScriptEngines it can find. Each engine identifies the scripting languages it supports. If a matching language isn't specified in the properties than that engine would be discarded.

Log4j 2 3.x will also come with a file named log4j2.default.component.yml that declares the default values for all properties except for log4j2.debug, log4j2.contextSelector, log4j2.ignoreTCL, and log4j2.forceTCLOnly whose values apply to all usages of Log4j in the JVM. The file below will be included in Log4j 2 to contain the default values.

{
	"log4j2": {
		"*": {
			"AsyncLogger": {
				"exceptionHandler": "",
				"ringBufferSize": "caclculate",
				"waitStrategy": "Timeout",
				"timeout": 10,
				"sleepTimeNS": 100,
				"retries": 200,
				"synchronizeEnqueWhenFull": true,
				"threadNameStrategy": "CACHED",
				"queueFullPolicy": "",
				"discardThreshold": "INFO",
				"formatMsg": false
			},
			"Configuration": {
				"location": "",
				"mergeStrategy": "org.apache.logging.log4j.core.config.composite.DefaultMergeStrategy",
				"configurationFactory": "org.apache.logging.log4j.core.config.ConfigurationFactory",
				"clock": "org.apache.logging.log4j.core.util.SystemClock",
				"level": "ERROR",
                "allowedProtocols": ["HTTPS"]
			},
			"GC": {
				"enableThreadLocals": true,
				"enableDirectEncoders": true,
				"intialReusableMsgSize": 128,
				"maxReusableMsgSize": 518,
				"layoutStringBuilderMaxSize": 2048,
				"unboxRingBufferSize": 32
			},
			"Jansi": {
				"enabled": "true"
			},
			"JMX": {
				"enabled": "false",
				"notifyAsync": "calculate"
			},
			"JNDI": {
				"enableContextSelector": "false",
				"enableJdbc": "false",
				"enableJMS": "false",
				"enableLookup": "false"
			},
			"JUL": {
				"loggerAdapter": "org.apache.logging.log4j.jul.ApiLoggerAdapter"
			},
			"LoggerContext": {
				"logEventFactory": "org.apache.logging.log4j.core.impl.DefaultLogEventFactory",
				"loggerContextFactory": "org.apache.logging.log4j.simple.SimpleLoggerContextFactory",
				"shutdownHookEnabled": "true",
				"shutdownCallbackRegistry": "org.apache.logging.log4j.core.util.DefaultShutdownCallbackRegistry",
				"stackTraceOnStart": false
			},
			"Message": {
				"messageFactory": "org.apche.logging.log4j.message.ParameterizedMessagefactory",
				"flowMessageFactory": "org.apache.logging.log4j.message.DefaultFlowMessageFactory",
				"jsonFormatterMaxDepth": 8
			},
			"Script": {
				"enableLanguages": []
			},
			"SimpleLogger": {
				"showContextMap": false,
				"showLogName": false,
				"showShortLogName": true,
				"showDateTime": false,
				"dateTimeFormat": "yyyy/MM/dd HH:mm:ss:SSS zzz",
				"logFile": "systemerr",
				"logLevel": "ERROR",
				"{loggerName}.level": "",
				"statusLoggerLevel": "ERROR"
			},
			"StatusLogger": {
				"defaultStatusLevel": "ERROR",
				"statusLoggerLevel": "WARN",
				"entries": 200,
				"dateFormat": ""
			},
			"ThreadContext": {
				"enabled": "true",
				"enableMap": "true",
				"enableStack": "false",
				"inheritable": "false",
				"garbageFree": "false",
				"initialCapacity": 16,
				"contextDataInjector": "org.apache.logging.log4j.core.ContextDataInjector"
			},
			"TransportSecurity": {
				"trustStoreLocation": "",
				"trustStorePassword": "",
				"trustStorePasswordFile": "",
				"trustStorePasswordEnvironmentVariable": "",
				"trustStoreType": "trustStoreKeyManagerFactoryAlgorithm",
				"keyStoreLocation": "",
				"keyStorePassword": "",
				"keyStorePasswordFile": "",
				"keytorePasswordEnvironmentVariable": "",
				"keyStoreType": "",
				"keyStoreKeyManagerFactoryAlgorithm": ""
			},
			"UUID": {
				"sequence": ""
			},
			"Web": {
				"enableWebApp": "calculate"
			}
		}
	}
}



11 Comments

  1. Matt Sicker

    This sounds like a good proposal. Do the component names need to be a single "normalized" term, or can they be multiple words like the property name is?

  2. Ralph Goers

    They need to be tokens valid in a JSON element name or property name and cannot contain a "." or "=".

  3. Gary D. Gregory

    How do I set a property for all logger contexts? Or for only the "root logger"?

    1. Ralph Goers

      See the update I just made to show the default settings. There are no settings for the Root Logger. That is solely managed in the Log4j 2 configuration file as it always has been.

  4. Volkan Yazici

    The proposal for the structure of a Log4j 2 property is log42.{LoggerContextName}.{ComponentName}.{property}.

    I guess here log42  is a typo.

    To keep the documentation from becoming a mess the current properties in 2.x would NOT be supported in 3.0.

    I have really mixed feelings about this statement. When I wanted to drop YAML, XML, JSON layout, and/or Liquibase adapter, I was objected with backward compatibility concerns. Yet here it is perfectly fine to simply destroy the compatibility to ease the documentation efforts. I just don't get it.

    I think we should (and can easily) support backward compatibility of properties. My suggestion would be to define aliases, use only the main property name in the manual, and create a page where we list all the aliases.

    Putting these details aside, I very much liked the idea.

    Given OWASP has offered assistance, I would suggest requesting review from experts before putting this in a release. It would be a shame if something that is supposed to lock down a behavior is not really doing its job.

  5. Ralph Goers

    Volkan Yazici Thanks for spotting the typo. I fixed that.

    The reasoning here is that users don't generally include system properties in their configuration or code. They would be able to upgrade without even compiling. All the stuff you are talking about breaks binary compatibility. 

    We already have aliases. So now we would have aliases of aliases. And we get TONS of questions about the ambiguities between the many variations of property names.

  6. Matt Sicker

    I've started centralizing all the various property names we used in various things. I've updated their names to the normalized forms, though we can start renaming them for v3 property names if we want to do that.

  7. Matt Sicker

    As for the default values file, there are some values that apply to the API and some that apply to other modules. Should each have their own default properties files, or should all the default values go in the API?

  8. Matt Sicker

    I wonder if it would be useful to include the "profiles" concept from Spring, too.

  9. Ralph Goers

    As I am implementing this I am having to think through some of the consequences of this:

    1. Application code should NEVER be looking for a property by including the "log4j2" constant or the context name token as doing so limits where the properties can be obtained from. For example, if the application were to specify "log4j2.*.Script.enableLanguages" then it could only be obtained from the global scope.
    2. Log4j core will supply an enum that all Log4j modules can make use of to find their properties. That should be leveraged. However, external components will not be able to take advantage of this and so must be able to provide either the key value as "component.key" or as two separate String parameters.
    3. This contextName isn't really usable when using the ClassLoaderContextSelector as the name is the name of the ClassLoader which is unpredictable.  With the Jndi
    4. Users should be able to supply system properties targeted at a specific context so system properties will be required to always include "log4j2.contextName] as the prefix to the property. Internally, these will targeted to the correct property set. i.e - properties will generally be reference via Properties objects without the leading "log4j2.contextName" but the Properties object will be attached to the targeted environment.
    5. Property files targeted at a context should generally not prefix values with "log4j2.contextname" as the context, and context name, is derived from how the file was located. These properties automatically override properties found at the global scope so a context name makes no sense here. Any context scoped files containing a key that starts with "log4j2." will be ignored with a message. Global property files may contain log4j2.contextName and if the contextName is not "*" then the properties will be applied as overrides to the context with the appropriate name.
    6. To be consistent global properties will continue to reside in log4j2.component.properties or log4j2.component.json. However, to avoid conflicts, when using the ClassLoaderContextSelector LoggerContext files will be required to be named log4j2.context.properties or log4j2.context.json and will be located using the ClassLoader associated with the LoggerContext. NamedContextSelectors will need to include the context name in the name of the file - such as log4j2.contextName.properties or log4j2.contextName.json but will still be located using the ClassLoader associated with the LoggerContext.

    Note that the above is as much notes to myself as anything else to make sure I implement something that will work.

  10. Ralph Goers

    More thoughts after doing some development. In Log4j 2.x environment variables were all uppercase with an _ used to separate "words". This doesn't work in with the properties enhancement because we separate the key into multiple tokens to represent the LoggerContext name, the component, and the key.  At first I tried converting all the keys to lower case and making everthing case insensitive, but then I realized that it would prevent things like the SpringPropertySource from working properly. Several of the unit tests fail due to this.  As such, I am now thinking the best solution might be to leave everything as case sensitive, including environment variables. Unix systems have no requirement that variables be limited to upper case letters, numbers, and underscore, however it seems that bash might make it a bit more difficult to work with variables that don't follow that convention.  I've seen mention that Unix/Linux/MacOS treat environment variables as case insensitive but that shouldn't matter for our purposes, so long as they maintain the case they were defined with. Windows not only allows mixed case but some standard system settings are mixed case.

    So, while for other property formats the syntax will be 

    log4j2.{contextName}.{componentName}.key

    for environment variables the syntax will instead be

    log4j2_{contextName}_{componentName}_key