Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stop updating endpoints when HealthCheckedEndpointGroup is closing #6063

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

ikhoon
Copy link
Contributor

@ikhoon ikhoon commented Jan 10, 2025

Motivation:

When HealthCheckedEndpointGroup is closing, the internally managed endpoints are cleared.

onUpdateHealth.accept(originalEndpoint.withAttrs(endpointAttributes), false);
In fact, the closing process is not a problem. However, when the endpoints change, the change is propagated to EndpointSelector. Some EndpointSelector implementations warn about the empty endpoints. The warning is not a real problem but a false positive alarm.
logger.warn("No valid endpoint with weight > 0. endpoints: {}", toShortString(endpoints));

For better compatibility with EndpointSelector implementations, I propose to stop additional endpoints from being updated when HealthCheckedEndpointGroup is closed. As a result, EndpointSelector will not updated with empty endpoints.

Modifications:

  • Stop updating endpoints when HealthCheckedEndpointGroup is closing

Result:

You no loger see a false positive warning log when HealthCheckedEndpointGroup is closing.

Motivation:

When `HealthCheckedEndpointGroup` is closing, it clears the internal
endpoints.
https://github.com/line/armeria/blob/011741dc51cf328b78488655e78ceca1cdc0a1b3/core/src/main/java/com/linecorp/armeria/client/endpoint/healthcheck/DefaultHealthCheckerContext.java#L135
In fact, the closing process is not a problem. However, when the
endpoints changes, the change is propagated to `EndpointSelector`.

Some `EndpointSelector` implementations warn about the empty endpoints.
https://github.com/line/armeria/blob/62da203bd9e11d669476e6999937352749cd7339/core/src/main/java/com/linecorp/armeria/client/endpoint/WeightedRoundRobinStrategy.java#L81
The warning is not a real problem but a false positive alarm.

For better compatibility with `EndpointSelector` implementations, I
propose to stop additional endpoints from being updated when
`HealthCheckedEndpointGroup` is closed. As a result, `EndpointSelector`
will not updated with empty endpoints.

Modifications:

- Stop updating endpoints when `HealthCheckedEndpointGroup` is closing

Result:

You no loger see a false positive warning log when
`HealthCheckedEndpointGroup` is closing.
@ikhoon ikhoon added the defect label Jan 10, 2025
@ikhoon ikhoon added this to the 1.32.0 milestone Jan 10, 2025
@@ -303,10 +303,17 @@ private void updateHealth(Endpoint endpoint, boolean health) {

// Each new health status will be updated after initialization of the first context group.
if (updated && initialized) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think of not calling the updateHealth at all from the DefaultHealthCheckerContext?

private DefaultHealthCheckerContext newCheckerContext(Endpoint endpoint) {
    return new DefaultHealthCheckerContext(endpoint, port, protocol, clientOptions, retryBackoff,
                                           this::updateHealth, this);
}

DefaultHealthCheckerContext(Endpoint endpoint, int port, SessionProtocol protocol,
                            ClientOptions clientOptions, Backoff retryBackoff,
                            BiConsumer<Endpoint, Boolean> onUpdateHealth,
                            ListenableAsyncCloseable listenableAsyncCloseable) {
  ...
}

// Use listenableAsyncCloseable.isClosing( ) to check if the HealthCheckedEndpointGroup is closing or not.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized that there's no heavy logic in the DefaultHealthCheckerContext.
So probably just check isClosing() in the updateHealth() method would be enough.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants