This Confluence has been LDAP enabled, if you are an ASF Committer, please use your LDAP Credentials to login. Any problems file an INFRA jira ticket please.

Child pages
  • Kafka Improvement Proposals
539 more child pages

Versions Compared


  • This line was added.
  • This line was removed.
  • Formatting was changed.


KIP-59: Proposal for a kafka broker commandSent emails to Dev discussion group. Work tracked under KAFKA-3663.
KIP-125: ZookeeperConsumerConnector to KafkaConsumer Migration and Rollback
KIP-131 - Add access to OffsetStorageReader from SourceConnector
KIP-135 : Send of null key to a compacted topic should throw non-retriable error back to user
KIP 141 - ProducerRecord & SourceRecord: Add timestamp constructors
KIP-142: Add ListTopicsRequest to efficiently list all the topics in a cluster
KIP-148: Add a connect timeout for client
KIP-158: Kafka Connect should allow source connectors to set topic-specific settings for new topics
KIP-159: Introducing Rich functions to Streams
KIP-166 - Add a tool to make amounts of replicas and leaders on brokers balanced
KIP-169 - Lag-Aware Partition Assignment Strategy
KIP-178: Size-based log directory selection strategy

KIP-185: Make exactly once in order delivery the default producer setting

KIP-193: Add SchemaBuilder.from(Schema)
KIP-199: Add Kafka Connect offset tool
KIP-201: Rationalising Policy interfaces
KIP-209: Connection String Support
KIP-216: IQ should throw different exceptions for different errors
KIP-217: Expose a timeout to allow an expired ZK session to be re-created
KIP-221: Enhance KStream with Connecting Topic Creation and Repartition HintDiscussion
KIP-228 Negative record timestamp supportVoting in progress
KIP-234: add support for getting topic defaults from AdminClient
KIP-236: Interruptible Partition Reassignment Discussion
KIP-240: AdminClient.listReassignments() AdminClient.describeReassignments()
KIP-242: Mask password in Kafka Connect Rest API response
KIP-250 Add Support for Quorum-based Producer Acknowledgment
KIP-252 - Extend ACLs to allow filtering based on ip ranges and subnets
KIP-253: Support in-order message delivery with partition expansion Discussion
KIP-254: JsonConverter Exception Handeling
KIP-259: Improve Streams DSL Timestamp Propagation Semantics
KIP-260: add primary join operation for Stream-Stream join (WIP)Draft
KIP-264: Add a consumer metric to record raw fetch sizeVoting in progress
KIP-271: Add NetworkClient redirectorDiscussion
KIP-273: Kafka to support using ETCD beside ZookeeperDiscussion
KIP-275 - Indicate "isClosing" in the SinkTaskContextVoting in progress
KIP-280: Enhanced log compactionDiscussion

KIP-291: Separating controller connections and requests from the data plane

KIP-293 Add new metrics for consumer/replication fetch requestsVoting in progress
KIP-296: Add connector level configurability for producer/consumer client configs Discussion
KIP-301: Schema Inferencing for JsonConverterDiscussion
KIP-304: Connect runtime mode improvements for container platformsDiscussion
KIP-314: KTable to GlobalKTable Bi-directional JoinDiscussion
KIP-315: Stream Join Sticky AssignorDiscussion
KIP-316: Command-line overrides for ConnectDistributed worker propertiesDiscussion
KIP-317: Add end-to-end data encryption functionality to Apache KafkaDiscussion
KIP-325: Extend Consumer Group Command to Show Beginning OffsetsVoting in Progress345
KIP-326: Schedulable KTable as Graph sourceDiscussion
KIP-333: Add faster mode of rebalancingDiscussion
KIP-334 - Include partitions in exceptions raised during consumer record deserialization/validation
KIP-335: Consider configurations for KafkaStreams


KIP-347: Enable batching in FindCoordinatorRequestDiscussion
KIP-348 Eliminate null from SourceTask#poll()
KIP-350: Allow to take brokerid as parameter to show partitions associated with it
KIP-356: Add KafkaConsumer fetch-error-rate and fetch-error-total metricsDiscussion
KIP-360: Improve handling of unknown producerDiscussion
KIP-362: Support Dynamic Session WindowDiscussion
KIP-363: Allow performance tools to print final results to output fileDiscussion
KIP-369: Alternative Partitioner to Support "Always Round-Robin" SelectionAccepted
KIP-370: Remove Orphan PartitionsDiscussion
KIP-373: Allow users to create delegation tokens for other usersDiscussion
KIP-375: Kafka Clients - make Metadata#TOPIC_EXPIRY_MS configurableDiscussion
KIP-378: Enable Dependency Injection for Kafka Streams handlersDiscussion

KIP-379: Multiple Consumer Group Management

KIP-381: Connect: Tell about records that had their offsets flushed in callback

Voting in progress (restarted 18th January 2019, due to no votes in first attempt)

KIP-383:  Pluggable interface for SSL FactoryVoting in progress
KIP-384: Add config for incompatible changes to persistent metadataDiscussion
KIP-385: Avoid throwing away prefetched dataDiscussion
KIP-387: Fair Message Consumption Across Partitions in KafkaConsumer


KIP-388: Add observer interface to record request and responseDiscussion
KIP-389: Introduce a configurable consumer group size limitAccepted
KIP-390: Allow fine-grained configuration for compressionDiscussion, JIRA exists with pull-request
KIP-391: Allow Producing with Offsets for Cluster ReplicationDiscussion, JIRA exists with pull-request
KIP-392: Allow consumers to fetch from closest replicaDiscussion
KIP-395: Encypt-then-MAC Delegation token metadata

KIP-396: Add Commit/List Offsets Operations to AdminClient

KIP-397: Add methods to override fetch offsets based on timestampDiscussion
KIP-398: Support reading trust store from classpathDiscussion
KIP-399: Extend ProductionExceptionHandler to cover serialization exceptionsDiscussion
KIP-400: Improve exit status in case of errors in ConsoleProducerDiscussion
KIP-401: TransformerSupplier/ProcessorSupplier StateStore connectingVoting in progress
KIP-403: Increase ProducerPerformance precision by using nanoTimeDraft
KIP-405: Kafka Tiered Storage


KIP-406: GlobalStreamThread should honor custom offset policy.Discussion
KIP-407: Kafka Connect support override worker kafka api configuration with connector configuration that post by rest api
KIP-408: Add asynchronous processing to Kafka StreamsDiscussion

KIP-409: Allow creating under-replicated topics and partitions

KIP-410: Add metric for request handler thread pool utilization by request type

KIP-416: Notify SourceTask of ACK'd offsets, metadataDiscussion
KIP-418: A method-chaining way to branch KStreamDiscussion, JIRA exists with pull-request
KIP-419: Safely notify Kafka Connect SourceTask is stoppedVoting in progress, JIRA exists with pull request
KIP-421: Support resolving externalized secrets in AbstractConfigAccepted
KIP-422: Add support for client quota configuration in the Kafka Admin ClientDiscussion, JIRA exists with pull-request
KIP-423: Add JoinReason to Consumer Join Group Protocol


KIP-424: Allow suppression of intermediate events based on wall clock timeDiscussion
KIP-426: Persist Broker Id to ZookeeperDiscussion
KIP-431: Support of printing additional ConsumerRecord fields in DefaultMessageFormatterVoting in progress, JIRA exists with pull-request
KIP-434: Add Replica Fetcher and Log Cleaner Count MetricsDiscussion
KIP-435: Incremental Partition ReassignmentDiscussion
KIP-437: Custom replacement for MaskField SMTVoting in progress, JIRA exists with pull-request
KIP-438: Expose task, connector IDs in Connect APIDiscussion
KIP-439: Cleanup built-in Store interfacesDiscussion
KIP-440: Extend Connect Converter to support headersVoting in progress
KIP-444: Augment metrics for Kafka StreamsDiscussion
KIP-448: Add State Stores Unit Test Support to Kafka Streams Test UtilsDiscussion
KIP-450: Sliding Window Aggregations in the DSLDiscussion
KIP-452: Tool to view cluster statusDiscussion, JIRA exists
KIP-455: Create an Administrative API for Replica ReassignmentDiscussion
KIP-456: Helper classes to make it simpler to write test logic with TopologyTestDriver


KIP-457: Add DISCONNECTED status to Kafka StreamsDiscussion 
KIP-459: Improve KafkaStreams#closeDiscussion
KIP-463: Auto-configure non-default Serdes passed alongside the TopologyBuilderUnder discussion
KIP-468: Avoid decompression of record when validate record at server in the scene of inPlaceAssignment .Under discussion
KIP-470: TopologyTestDriver test input and output usability improvementsUnder discussion
KIP-471: Expose RocksDB Metrics in Kafka Streams

Under discussion

KIP-466: Add support for List<T> serialization and deserializationUnder Discussion
KIP-473: Enable KafkaLog4JAppender to use SASL Authentication Callback HandlersUnder Discussion 
KIP-474: To deprecate WindowStore#put(key, value)Under Discussion
KIP-475: New Metric to Measure Number of Tasks on a ConnectorUnder DiscussionVoting
KIP-476: Add Java AdminClient InterfaceUnder Discussion