Status

Current state: Under Discussion

Discussion thread: here

JIRA: KAFKA-19515 

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

In the controller, we can configure the listeners setting as CONTROLLER://0.0.0.0:9093, which means it will bind to all network interfaces. If advertised.listeners is not explicitly set, the controller will automatically infer it from the listeners value. However, the behavior is different for the broker. If we set listeners to PLAINTEXT://0.0.0.0:9093, the broker throws an IllegalArgumentException instead of inferring the value of advertised.listeners.

This KIP aim to provide a consistent and predictable experience. Therefore, I believe we should align this behavior across controller and broker nodes. This alignment has several benefits:

• No longer need to remember which node types require advertised.listeners to be explicitly set.

• This allows users to configure broker properties with greater flexibility.

Public Interfaces

• org.apache.kafka.network.SocketServerConfigs 

Proposed Changes

The valid value of advertised.listeners config will changed as following table:

We can add the following logic to filter 0.0.0.0 related listeners when init broker side advertised listeners.

def effectiveAdvertisedBrokerListeners: Seq[Endpoint] = {
  // Use advertised listeners if defined, fallback to listeners otherwise
  val advertisedListenersProp = getString(SocketServerConfigs.ADVERTISED_LISTENERS_CONFIG)
  val advertisedListeners = if (advertisedListenersProp != null) {
    CoreUtils.listenerListToEndPoints(advertisedListenersProp, effectiveListenerSecurityProtocolMap, requireDistinctPorts = false)
  } else {
    listeners
  }
  // Only expose broker listeners
  advertisedListeners.filterNot(l => controllerListenerNames.contains(l.listener))
    .map(endpoint =>
      if (endpoint.host == "0.0.0.0") {
        new Endpoint(endpoint.listener, endpoint.securityProtocol, null, endpoint.port)
      } else {
        endpoint
      }
    )
}

Compatibility, Deprecation, and Migration Plan

Since this is a newly introduced behavior, there are no compatibility concerns.

Test Plan

All existing tests should continue to pass, and we will add tests to cover the new behavior for empty advertised.listeners.

Rejected Alternatives

Deprecated the controller allow empty advertised.listeners 

This is a user-friendly design, so instead of prohibiting this behavior, it would be better to make the broker's behavior consistent as well. Therefore, rejecting it might not be the optimal solution.