Log4j 2 2.x uses the PropertiesUtil class in log4j-api to access properties. It was enhanced in Log4j 2.10.0 to "normalize" property names. While this helped, with the growth in the number of properties Log4j uses managing them is getting harder.
We also desire to be able to have components enabled or disabled by default but to provide settings to override the default. With Log4j 2 3.x being much more modularized enablement of many things can be controlled simply by the presence of the jar that contains the relevant functionality. However, it is sometimes the case that we will want individual plugins to require specific enablement to be activated. The structure (or lack of) the current set of properties makes this difficult.
This proposal is to require all properties to follow a more well defined structure. The proposal for the structure of a Log4j 2 property is log4j2.{LoggerContextName}.{ComponentName}.{property}. To keep the documentation from becoming a mess the current properties in 2.x would NOT be supported in 3.0.
The LoggerContext has always been allowed to have a name. it currently is internally generated and used as the unique key to locate it based on its associated ClassLoader. This is unfortunate as there are cases where users would like to specify properties that apply only to a specific LoggerContext. This proposal would create a new private LoggerContext field named contextKey that is used in place of the contextName as the key in the LoggerContext Map. The various API layers would be modified to accept a contextName field when he LoggerContext is created. The context name will be immutable.
This will allow a configuration like the one shown below. log4j2 at the top level identifies the file as containing log4j properties. The element at the next level down is the name of the LoggerContext the enclosed properties apply to. A "*" as the LoggerContext name indicates the settings are the default values. Within that properties are grouped by the components they apply to. Note that the term "component" is used loosely here.
Note that JsonTemplateLayout includes a JsonReader. If that is moved into log4j-core we should be able to support JSON configuration and properties without any additional dependencies. In addition, our Spring Boot support will allow the user to specify the JSON below in the application's bootstrap.yml file as an alternative to log4j2.component.properties and log4j2.component.yml.
{ "log4j2": {
"My-App": {
"JNDI": {
"enableJMS": "true"
}
"Script": {
"enableLanguages": [
"Groovy",
"JavaScript"
]
}
"Configuration": {
"mergeStrategy": "com.acme.log4j.CustomMergeStrategy",
"location": "classpath:log4j2-My-App.xml",
"statusLoggerLevel": "debug"
}
}, "*": {
"StatusLogger": {
"defaultStatusLevel": "Info"
}
}}
Of course, the equivalent would also be available as properties, although the property names will change from their current values
log4j2.My-App.Jms.enableJndi=true log4j2.My-App.Script.enableLanguages=Groovy,Javascript log4j2.My-App.Configuration.mergeStrategy=com.acme.log4j.CustomMergeStrategy log4j2.My-App.Configuration.location=classpath:log4j2-My-App.xml log4j2.My-App.Configuration.statusLoggerLevel=debug log4j2.*.StatusLogger.defaultStatusLevel=Info
Using this format allows restrictions to be placed on plugins:
@RequireProperty(component = "Lookup", property = "enableJndi", value = "true")
@Plugin(name = "jndi", category = StrLookup.CATEGORY)
public class JndiLookup extends AbstractLookup {}
This would prevent the Plugin from being loaded unless the requirement is met.
In the case of Scripting we would want the ScriptManager to be disabled unless one or more scripting language are enabled. In the case of the ScriptManager it is loaded by the ScriptManagerFactory in Log4j 2 3.x, which is loaded via ServiceLoader. The ScriptManager then locates all the ScriptEngines it can find. Each engine identifies the scripting languages it supports. If a matching language isn't specified in the properties than that engine would be discarded.
Log4j 2 3.x will also come with a file named log4j2.default.component.yml that declares the default values for all properties except for log4j2.debug, log4j2.contextSelector, log4j2.ignoreTCL, and log4j2.forceTCLOnly whose values apply to all usages of Log4j in the JVM. The file below will be included in Log4j 2 to contain the default values.
{
"log4j2": {
"*": {
"AsyncLogger": {
"exceptionHandler": "",
"ringBufferSize": "caclculate",
"waitStrategy": "Timeout",
"timeout": 10,
"sleepTimeNS": 100,
"retries": 200,
"synchronizeEnqueWhenFull": true,
"threadNameStrategy": "CACHED",
"queueFullPolicy": "",
"discardThreshold": "INFO",
"formatMsg": false
},
"Configuration": {
"location": "",
"mergeStrategy": "org.apache.logging.log4j.core.config.composite.DefaultMergeStrategy",
"configurationFactory": "org.apache.logging.log4j.core.config.ConfigurationFactory",
"clock": "org.apache.logging.log4j.core.util.SystemClock",
"level": "ERROR",
"allowedProtocols": ["HTTPS"]
},
"GC": {
"enableThreadLocals": true,
"enableDirectEncoders": true,
"intialReusableMsgSize": 128,
"maxReusableMsgSize": 518,
"layoutStringBuilderMaxSize": 2048,
"unboxRingBufferSize": 32
},
"Jansi": {
"enabled": "true"
},
"JMX": {
"enabled": "false",
"notifyAsync": "calculate"
},
"JNDI": {
"enableContextSelector": "false",
"enableJdbc": "false",
"enableJMS": "false",
"enableLookup": "false"
},
"JUL": {
"loggerAdapter": "org.apache.logging.log4j.jul.ApiLoggerAdapter"
},
"LoggerContext": {
"logEventFactory": "org.apache.logging.log4j.core.impl.DefaultLogEventFactory",
"loggerContextFactory": "org.apache.logging.log4j.simple.SimpleLoggerContextFactory",
"shutdownHookEnabled": "true",
"shutdownCallbackRegistry": "org.apache.logging.log4j.core.util.DefaultShutdownCallbackRegistry",
"stackTraceOnStart": false
},
"Message": {
"messageFactory": "org.apche.logging.log4j.message.ParameterizedMessagefactory",
"flowMessageFactory": "org.apache.logging.log4j.message.DefaultFlowMessageFactory",
"jsonFormatterMaxDepth": 8
},
"Script": {
"enableLanguages": []
},
"SimpleLogger": {
"showContextMap": false,
"showLogName": false,
"showShortLogName": true,
"showDateTime": false,
"dateTimeFormat": "yyyy/MM/dd HH:mm:ss:SSS zzz",
"logFile": "systemerr",
"logLevel": "ERROR",
"{loggerName}.level": "",
"statusLoggerLevel": "ERROR"
},
"StatusLogger": {
"defaultStatusLevel": "ERROR",
"statusLoggerLevel": "WARN",
"entries": 200,
"dateFormat": ""
},
"ThreadContext": {
"enabled": "true",
"enableMap": "true",
"enableStack": "false",
"inheritable": "false",
"garbageFree": "false",
"initialCapacity": 16,
"contextDataInjector": "org.apache.logging.log4j.core.ContextDataInjector"
},
"TransportSecurity": {
"trustStoreLocation": "",
"trustStorePassword": "",
"trustStorePasswordFile": "",
"trustStorePasswordEnvironmentVariable": "",
"trustStoreType": "trustStoreKeyManagerFactoryAlgorithm",
"keyStoreLocation": "",
"keyStorePassword": "",
"keyStorePasswordFile": "",
"keytorePasswordEnvironmentVariable": "",
"keyStoreType": "",
"keyStoreKeyManagerFactoryAlgorithm": ""
},
"UUID": {
"sequence": ""
},
"Web": {
"enableWebApp": "calculate"
}
}
}
}
11 Comments
Matt Sicker
This sounds like a good proposal. Do the component names need to be a single "normalized" term, or can they be multiple words like the property name is?
Ralph Goers
They need to be tokens valid in a JSON element name or property name and cannot contain a "." or "=".
Gary D. Gregory
How do I set a property for all logger contexts? Or for only the "root logger"?
Ralph Goers
See the update I just made to show the default settings. There are no settings for the Root Logger. That is solely managed in the Log4j 2 configuration file as it always has been.
Volkan Yazici
I guess here
log42is a typo.I have really mixed feelings about this statement. When I wanted to drop YAML, XML, JSON layout, and/or Liquibase adapter, I was objected with backward compatibility concerns. Yet here it is perfectly fine to simply destroy the compatibility to ease the documentation efforts. I just don't get it.
I think we should (and can easily) support backward compatibility of properties. My suggestion would be to define aliases, use only the main property name in the manual, and create a page where we list all the aliases.
Putting these details aside, I very much liked the idea.
Given OWASP has offered assistance, I would suggest requesting review from experts before putting this in a release. It would be a shame if something that is supposed to lock down a behavior is not really doing its job.
Ralph Goers
Volkan Yazici Thanks for spotting the typo. I fixed that.
The reasoning here is that users don't generally include system properties in their configuration or code. They would be able to upgrade without even compiling. All the stuff you are talking about breaks binary compatibility.
We already have aliases. So now we would have aliases of aliases. And we get TONS of questions about the ambiguities between the many variations of property names.
Matt Sicker
I've started centralizing all the various property names we used in various things. I've updated their names to the normalized forms, though we can start renaming them for v3 property names if we want to do that.
Matt Sicker
As for the default values file, there are some values that apply to the API and some that apply to other modules. Should each have their own default properties files, or should all the default values go in the API?
Matt Sicker
I wonder if it would be useful to include the "profiles" concept from Spring, too.
Ralph Goers
As I am implementing this I am having to think through some of the consequences of this:
Note that the above is as much notes to myself as anything else to make sure I implement something that will work.
Ralph Goers
More thoughts after doing some development. In Log4j 2.x environment variables were all uppercase with an _ used to separate "words". This doesn't work in with the properties enhancement because we separate the key into multiple tokens to represent the LoggerContext name, the component, and the key. At first I tried converting all the keys to lower case and making everthing case insensitive, but then I realized that it would prevent things like the SpringPropertySource from working properly. Several of the unit tests fail due to this. As such, I am now thinking the best solution might be to leave everything as case sensitive, including environment variables. Unix systems have no requirement that variables be limited to upper case letters, numbers, and underscore, however it seems that bash might make it a bit more difficult to work with variables that don't follow that convention. I've seen mention that Unix/Linux/MacOS treat environment variables as case insensitive but that shouldn't matter for our purposes, so long as they maintain the case they were defined with. Windows not only allows mixed case but some standard system settings are mixed case.
So, while for other property formats the syntax will be
log4j2.{contextName}.{componentName}.keyfor environment variables the syntax will instead be
log4j2_{contextName}_{componentName}_key