This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Child pages
  • KIP-33 - Add a time based log index

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Currently time based log rolling is based on the creating time of the log segment. With this KIP, the time based rolling would be changed to only based on the message timestamp of . More specifically, if the first message in the log segment . has a timestamp, A new log segment will be rolled out if current time timestamp in the message about to be appended is greater than the timestamp of the first message in the segment + log.roll.ms. When message.timestamp.type=CreateTime, user should set max.message.time.difference.ms appropriately together with log.roll.ms to avoid frequent log segment roll out.

During the migration phase, if the first message in a segment does not have a timestamp, the log rolling will still be based on the (current time - create time of the segment).

Search message by timestamp

...