Status

Current state: Accepted

Discussion thread: here 

JIRA: here 

Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).

Motivation

RLMM allows to pass properties specific to producer/consumer with `remote.log.metadata.producer.` and `remote.log.metadata.consumer.` prefixes.

However, there is no way to set configs specifically for the admin client without affecting producers or consumers.

Drawbacks of the current approach:

Without the remote.log.metadata.admin. config prefix, admin client configs must be set using the common remote.log.metadata.common.client. prefix. This poses several limitations:

  1. Lack of isolation: Admin-specific configs cannot be separated from general client configs, potentially leading to conflicts or suboptimal settings for admin operations.

  2. Limited flexibility: Admin clients may require different configs (e.g., timeouts, retries, security settings) compared to producers/consumers, but the current approach forces shared settings.

  3. Operational constraints: In environments where admin operations need stricter security or different network configs, the inability to isolate these settings creates operational challenges.

Public Interfaces

  • Add a new prefix in `TopicBasedRemoteLogMetadataManagerConfig`
TopicBasedRemoteLogMetadataManagerConfig.java
public static final String REMOTE_LOG_METADATA_ADMIN_PREFIX = "remote.log.metadata.admin.";

Config Precedence:

The remote.log.metadata.admin. prefix will take precedence over remote.log.metadata.common.client. The config resolution order will be:

  1. remote.log.metadata.admin. (highest priority)

  2. remote.log.metadata.common.client. (fallback)

  3. Default values (lowest priority)

Proposed Changes

The admin prefix will be added to the TopicBasedRemoteLogMetadataManagerConfig, allowing users to set config exclusively for the admin client.

Compatibility, Deprecation, and Migration Plan

This change is fully backwards compatible.

Test Plan

The patch will include unit tests to ensure full coverage.

Rejected Alternatives

None.

  • No labels