This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Child pages
  • KIP-284: Set default retention ms for Streams repartition topics to Long.MAX_VALUE
Skip to end of metadata
Go to start of metadata

Status

Current state: Accepted 

Discussion threadMailing list

JIRA: KAFKA-6535

Release : planned for 1.2

Motivation

After KIP-220 / KIP-204, repartition topics in Streams are transient, so it is better to set its default retention to infinity to allow any records be pushed to it with old timestamps (think: bootstrapping, re-processing) and just rely on the purging API to keeping its storage small.

Public Interfaces

The users may take in consideration that there is no limited duration of the retention, so it is up to them to manage the purging of the storage.

Proposed Changes

In RepartitionTopicConfig we have a few default overrides for repartition topic configs, we should just add the override for TopicConfig.RETENTION_MS_CONFIG to set it to Long.MAX_VALUE. This still allows users to override themselves if they want via StreamsConfig.TOPIC_PREFIX.

  • No labels