Repository navigation
Fix service re-registration after Consul connection failures - #1038
ryanjbaxter merged 3 commits into
Conversation
Signed-off-by: 98001yash <yashchauhan.gaya@gmail.com>
ryanjbaxter
left a comment
There was a problem hiding this comment.
An error that happens repeatedly will be noisy by logging the stacktrace, maybe log the stacktrace at debug level?
| this.ttlScheduler.client.agentServiceRegister( | ||
| this.ttlScheduler.discoveryProperties.getAclToken(), registeredService); | ||
| } | ||
| catch (ResourceAccessException registrationException) { |
There was a problem hiding this comment.
If Consul is reachable but the re-register call returns a 5xx or 403, HttpStatusCodeException still escapes from here, while a connection failure is now handled. Would catching RestClientException be more consistent?
Also, there’s no test for this path.
|
Since |
Signed-off-by: 98001yash <yashchauhan.gaya@gmail.com>
|
Addressed the review feedback by handling |
ryanjbaxter
left a comment
There was a problem hiding this comment.
Could we add a test for when failFast is true? If that is true we should not retry IMO
| } | ||
| catch (ResourceAccessException e) { | ||
| log.warn("Unable to send heartbeat to Consul for service " + this.serviceId | ||
| + ". Will retry on the next heartbeat."); |
There was a problem hiding this comment.
Appending e.getMessage to the warn log would be useful to at a high level see what the reason for the failure was yet not pollute the log with the stacktrace
| log.warn("Failfast is false. Error registering service with consul: " + reg.getService(), e); | ||
| } | ||
|
|
||
| NewService service = reg.getService(); |
There was a problem hiding this comment.
Why was this moved outside of the try?
There was a problem hiding this comment.
The scheduling was moved outside the try-catch to ensure that heartbeat retries are still scheduled when the initial registration fails and failFast=false. Otherwise, the registration exception would prevent scheduling, leaving the service without a retry mechanism. When failFast=true, the exception is rethrown before reaching the scheduling logic, so no heartbeat retry is scheduled.
Signed-off-by: 98001yash <yashchauhan.gaya@gmail.com>
Summary
Fix service re-registration behavior when Consul is unavailable during initial service registration or subsequent heartbeat attempts.
Changes
ResourceAccessExceptionduring heartbeat execution so transient connection failures do not prevent subsequent heartbeat attempts.Testing
ConsulHeartbeatTaskTestsConsulServiceRegistryRegistrationTests.\mvnw.cmd -pl spring-cloud-consul-discovery -am testgit diff --checkAll tests passed: 145 tests run, 0 failures, 0 errors, 1 skipped.
Related Issue
Fixes #774