Current state: Under discussion.
Discussion thread: here (Not happening yet)
Currently, clients would fail if DNS resolution fails. The application owner will either need to implement retry logic or manually restart the application. This is inconvenient and hard to handle because:
Failed DNS resolution throws a ConfigException, which is not descriptive of the actual problem (the message is fine, but the exception type is misleading); unless the developer tries to parse and match the error message.
It can take minutes before the bootstrap server is registered to the DNS server, and it is reasonable to allow clients to continue to retry.
- Users can catch a NetworkException and retry. (Remove the ConfigException)
- Several logging (warn) around failing DNS resolution will be removed.
- DNS lookup will happen on the first poll.
Remove the DNS lookup in the client constructor and delegate this task to the NetworkClient poll method, which means the client's won't attempt to resolve for DNS upon starting.
Client Constructor: Only parse the bootstrap config and validate its format there
Bootstrapping should now occur in the poll method before attempting to update the metadata. This includes resolving the addresses and bootstrapping the metadata.
Throws and logs a NetworkException if DNS resolution fails.
Compatibility, Deprecation, and Migration Plan
Clients won’t attempt to resolve the bootstrap addresses upon initialization.
Clients won’t exit fatally if DNS resolution fails.
KafkaConsumer: Users will need to poll to retry the lookup if it failed.
KafkaAdminClient: Users will need to resend the request if failing.
KafkaProducer: The sender loop should already be polling continuously.
Failed DNS resolution will result in NetworkException
Test DNS resolution upon its initial poll
Test if the right exception type is thrown
Existing clients (Consumer, Producer, AdminClient)
Test successful bootstrapping upon retrying
Allow the application owner to specify a retry period. The clients will fail after exceeding the timeout. The default set to 0s, which makes retry an opt-in config.
Pros: Allows users to have more control over how long to retry
Cons: Require a new config; client instantiation can block.
No retry. Let the application owner handle the DNS resolution exception. This means we would still throw a DNSLookupException upon failing.
Pros: No additional config is needed
Cons: This is a behavioral change, and the application owner might need to rewrite the exception handling, i.e. catching the DNS failure logic.
- Not throwing an exception but letting NetworkClient retry on poll.
- Pros: No compatibility break. No additional exception handling logic, the network client will just log the error and continue to retry upon the next poll
- Cons: The discussion thread mentioned that it wouldn't fail upon starting.