ISR information returned from metadata requests can be inconsistent with that stored/ returned from Zookeeper. Metadata is only propagated from controller to the brokers when the controller changes the leader or the ISR. However, when a follower catches up, the leader (not the controller) adds it back to ISR and updates ZK. This information is not propagated to all brokers.
Current state: Discarded
Discussion thread: here
JIRA: KAFKA-1367
Please keep the discussion on the mailing list rather than commenting on the wiki (wiki discussions get unwieldy fast).
In current design, ISR information in topic metadata response from brokers can be out of sync. The incorrect information can be a problem for the clients relying on this information.
Will update this section once we have decided on the solution we want to go ahead with.
Briefly list any new interfaces that will be introduced as part of this proposal or any existing interfaces that will be removed or changed. The purpose of this section is to concisely call out the public contract that will come along with this feature.
A public interface is any change to the following:
Binary log format
The network protocol and api behavior
Any class in the public packages under clientsConfiguration, especially client configuration
org/apache/kafka/common/serialization
org/apache/kafka/common
org/apache/kafka/common/errors
org/apache/kafka/clients/producer
org/apache/kafka/clients/consumer (eventually, once stable)
Monitoring
Command line tools and arguments
TopicMetadataRequest's ISR information, which gets out of sync for a broker that gets bounced/ restarted, does not have any good known use case. It is proposed to remove ISR information from the TopicMetadataRequest. However, a new request, BrokerMetadataRequest, is proposed to be added. The new request will include ISR information for all the partitions that the broker leads.