io.servicetalk.traffic.resilience.http

Class TrafficResilienceHttpClientFilter.Builder

Object

TrafficResilienceHttpClientFilter.Builder

Enclosing class:

TrafficResilienceHttpClientFilter

public static final class TrafficResilienceHttpClientFilter.Builder
extends Object

A TrafficResilienceHttpServiceFilter instance builder.

Constructor Summary

Constructors

Constructor and Description
Builder(Supplier<CapacityLimiter> capacityLimiterSupplier) A TrafficResilienceHttpClientFilter with no partitioning schemes.
Builder(Supplier<Function<HttpRequestMetaData,CapacityLimiter>> capacityPartitionsSupplier, boolean rejectNotMatched) A TrafficResilienceHttpClientFilter can support request partitioning schemes.

Method Summary

Modifier and Type	Method and Description
TrafficResilienceHttpClientFilter	build() Invoke to build an instance of TrafficResilienceHttpClientFilter filter to be used inside the HttpClientBuilder.
TrafficResilienceHttpClientFilter.Builder	capacityPartitions(Supplier<Function<HttpRequestMetaData,CapacityLimiter>> capacityPartitionsSupplier, boolean rejectNotMatched) Define CapacityLimiter partitions.
TrafficResilienceHttpClientFilter.Builder	circuitBreakerPartitions(Supplier<Function<HttpRequestMetaData,CircuitBreaker>> circuitBreakerPartitionsSupplier) Define CircuitBreaker to manage local or remote errors.
TrafficResilienceHttpClientFilter.Builder	classifier(Supplier<Function<HttpRequestMetaData,Classification>> classifier) Classification in the context of capacity management allows for hints to the relevant CapacityLimiter to be influenced on the decision-making process by the class of the request.
TrafficResilienceHttpClientFilter.Builder	dontForceOpenCircuitOnPeerCircuitRejections() When a peer rejects a request due to an open-circuit (see.
TrafficResilienceHttpClientFilter.Builder	dryRun(boolean dryRun) Use the resilience filter in dry-run mode.
TrafficResilienceHttpClientFilter.Builder	forceOpenCircuitOnPeerCircuitRejections(Supplier<Function<HttpResponseMetaData,Duration>> delayProvider, Executor executor) When a peer rejects a request due to an open-circuit (see.
TrafficResilienceHttpClientFilter.Builder	observer(TrafficResiliencyObserver observer) Provide an observer to track interactions of the filter and requests.
TrafficResilienceHttpClientFilter.Builder	onCancelTicketTerminal(Consumer<CapacityLimiter.Ticket> onCancellation) Ticket terminal callback override upon cancellation of the request operation.
TrafficResilienceHttpClientFilter.Builder	onErrorTicketTerminal(BiConsumer<CapacityLimiter.Ticket,Throwable> onError) Ticket terminal callback override upon erroneous completion of the request operation.
TrafficResilienceHttpClientFilter.Builder	peerUnavailableRejectionPredicate(Predicate<HttpResponseMetaData> rejectionPredicate) Peers can reject requests due to service unavailability.
TrafficResilienceHttpClientFilter.Builder	rejectionPolicy(ClientPeerRejectionPolicy policy) Peers can reject and exception due to capacity reasons based on their own principals and implementation details.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Builder

public Builder(Supplier<CapacityLimiter> capacityLimiterSupplier)

A TrafficResilienceHttpClientFilter with no partitioning schemes.

All requests will go through the CapacityLimiter.

Parameters:

capacityLimiterSupplier - The Supplier to create a new CapacityLimiter for each new filter created by this factory.

Builder

public Builder(Supplier<Function<HttpRequestMetaData,CapacityLimiter>> capacityPartitionsSupplier,
               boolean rejectNotMatched)

A TrafficResilienceHttpClientFilter can support request partitioning schemes.

A partition in the context of capacity management, is a set of requests that represent an application characteristic relevant to capacity, which can be isolated and have their own set of rules (ie. CapacityLimiter).

An example of a partition can be to represent each customer in a multi-tenant service. If an application wants to introduce customer API quotas, they can do so by identifying that customer through the HttpRequestMetaData and providing a different CapacityLimiter for that customer.

If a partitions doesn't return a CapacityLimiter for the given HttpRequestMetaData then the rejectNotMatched is evaluated to decide what the filter should do with this request. If true then the request will be rejected.

It's important that instances returned from this mapper are singletons and shared across the same matched partitions. Otherwise, capacity will not be controlled as expected, and there is also the risk for OutOfMemoryError.

Parameters:

capacityPartitionsSupplier - A Supplier to create a new Function for each new filter created by this factory. Function provides a CapacityLimiter instance for the given HttpRequestMetaData.

rejectNotMatched - Flag that decides what the filter should do when partitions doesn't return a CapacityLimiter.

Method Detail

capacityPartitions

public TrafficResilienceHttpClientFilter.Builder capacityPartitions(Supplier<Function<HttpRequestMetaData,CapacityLimiter>> capacityPartitionsSupplier,
                                                                    boolean rejectNotMatched)

Define CapacityLimiter partitions.

A partition in the context of capacity management, is a set of requests that represent an application characteristic relevant to capacity, which can be isolated and have their own set of rules (ie. CapacityLimiter).

An example of a partition can be to represent each customer in a multi-tenant service. If an application wants to introduce customer API quotas, they can do so by identifying that customer through the HttpRequestMetaData and providing a different CapacityLimiter for that customer.

If a partitions doesn't return a CapacityLimiter for the given HttpRequestMetaData then the rejectNotMatched is evaluated to decide what the filter should do with this request. If true then the request will be rejected.

It's important that instances returned from this mapper are singletons and shared across the same matched partitions. Otherwise, capacity will not be controlled as expected, and there is also the risk for OutOfMemoryError.

Parameters:

capacityPartitionsSupplier - A Supplier to create a new Function for each new filter created by this factory. Function provides a CapacityLimiter instance for the given HttpRequestMetaData.

rejectNotMatched - Flag that decides what the filter should do when partitions doesn't return a CapacityLimiter.

Returns:

this

classifier

public TrafficResilienceHttpClientFilter.Builder classifier(Supplier<Function<HttpRequestMetaData,Classification>> classifier)

Classification in the context of capacity management allows for hints to the relevant CapacityLimiter to be influenced on the decision-making process by the class of the request.

An example of classification, could be health checks that need to be given preference and still allowed a permit even under stress conditions. Another case, could be a separation of reads and writes, giving preference to the reads will result in a more available system under stress, by rejecting earlier writes.

The behavior of the classification and their thresholds could be different among different CapacityLimiter implementations, therefore the use of this API requires good understanding of how the algorithm in use will react for the different classifications.

Classification works within the context of a single Builder(Supplier, boolean) partition} and not universally in the filter.

It's worth noting that classification is strictly a hint and could be ignored by the CapacityLimiter.

Parameters:

classifier - A Supplier of a Function that maps an incoming HttpRequestMetaData to a Classification.

Returns:

this.

circuitBreakerPartitions

public TrafficResilienceHttpClientFilter.Builder circuitBreakerPartitions(Supplier<Function<HttpRequestMetaData,CircuitBreaker>> circuitBreakerPartitionsSupplier)

Define CircuitBreaker to manage local or remote errors.

The breakers can either be universal or follow any partitioning scheme (i.e., API / service-path, customer etc) but is recommended to follow similar schematics between service and client if possible for best experience.

The matching CircuitBreaker for a request can be forced opened due to a remote open circuit-breaker (i.e., HttpResponseStatus.SERVICE_UNAVAILABLE) dissallowing further outgoing requests for a fixed periods; forceOpenCircuitOnPeerCircuitRejections(Supplier, Executor).

Parameters:

circuitBreakerPartitionsSupplier - A Supplier to create a new Function for each new filter created by this factory. Function provides a CircuitBreaker instance for the given HttpRequestMetaData.

Returns:

this.

rejectionPolicy

public TrafficResilienceHttpClientFilter.Builder rejectionPolicy(ClientPeerRejectionPolicy policy)

Peers can reject and exception due to capacity reasons based on their own principals and implementation details. A TrafficResilienceHttpClientFilter can benefit from this input as feedback for the CapacityLimiter in use, that the request was dropped (i.e., rejected), thus it can also bring its local limit down to help with the overloaded peer. Since what defines a rejection/drop or request for backpressure is not universally common, one can define what response characteristics define that state.

It's important to know that if the passed rejectionPredicate tests true for a given HttpResponseMetaData then the operation is Single.failed(Throwable).

Out of the box if nothing custom is defined, the filter recognises as rejections requests with the following status codes:

• HttpResponseStatus.TOO_MANY_REQUESTS
• HttpResponseStatus.BAD_GATEWAY

Allowing retry, requests will fail with a DelayedRetryRequestDroppedException to support retrying mechanisms (like retry-filters or retry operators) to re-attempt the same request. Requests that fail due to capacity limitation, are good candidates for a retry, since we anticipate they are safe to be executed again (no previous invocation actually started) and because this maximizes the success chances.

Parameters:

policy - The ClientPeerRejectionPolicy that represents the peer capacity rejection behavior.

Returns:

this.

See Also:

ClientPeerRejectionPolicy.DEFAULT_PEER_REJECTION_POLICY

peerUnavailableRejectionPredicate

public TrafficResilienceHttpClientFilter.Builder peerUnavailableRejectionPredicate(Predicate<HttpResponseMetaData> rejectionPredicate)

Peers can reject requests due to service unavailability. A TrafficResilienceHttpClientFilter can benefit from this input as feedback for the CircuitBreaker in use. A similar exception can be generated locally as a result of that feedback, to help the active local CircuitBreaker to also adapt.

It's important to know that if the passed rejectionPredicate tests true for a given HttpResponseMetaData then the operation will be Single.failed(Throwable).

Out of the box if nothing custom is defined, the filter recognises as rejections requests with the following status codes:

• HttpResponseStatus.SERVICE_UNAVAILABLE

Parameters:

rejectionPredicate - The Function that resolves a response to a peer-rejection or not.

Returns:

this.

forceOpenCircuitOnPeerCircuitRejections

public TrafficResilienceHttpClientFilter.Builder forceOpenCircuitOnPeerCircuitRejections(Supplier<Function<HttpResponseMetaData,Duration>> delayProvider,
                                                                                         Executor executor)

When a peer rejects a request due to an open-circuit (see. peerUnavailableRejectionPredicate(Predicate)), the feedback can be used to also forcefully open the local request's CircuitBreaker. The local CircuitBreaker will close again once a delay period passes as defined/extracted through the delayProvider.

If the delay provided is not a positive value, then the CircuitBreaker will not be modified.

To disable this behaviour see dontForceOpenCircuitOnPeerCircuitRejections().

Parameters:

delayProvider - A function to provide / extract a delay in milliseconds for the CircuitBreaker to remain open.

executor - A Executor used to re-close the CircuitBreaker once the delay expires.

Returns:

this.

dontForceOpenCircuitOnPeerCircuitRejections

public TrafficResilienceHttpClientFilter.Builder dontForceOpenCircuitOnPeerCircuitRejections()

When a peer rejects a request due to an open-circuit (see. peerUnavailableRejectionPredicate(Predicate)), ignore feedback and leave local matching circuit-breake partition closed.

To opt-in for this behaviour see forceOpenCircuitOnPeerCircuitRejections(Supplier, Executor).

Returns:

this.

onErrorTicketTerminal

public TrafficResilienceHttpClientFilter.Builder onErrorTicketTerminal(BiConsumer<CapacityLimiter.Ticket,Throwable> onError)

Ticket terminal callback override upon erroneous completion of the request operation. Erroneous completion in this context means, that an error occurred as part of the operation or the rejectionPolicy(ClientPeerRejectionPolicy) triggered an exception. By default the terminal callback is CapacityLimiter.Ticket.failed(Throwable).

Parameters:

onError - Callback to override default ticket terminal event for an erroneous operation.

Returns:

this.

onCancelTicketTerminal

public TrafficResilienceHttpClientFilter.Builder onCancelTicketTerminal(Consumer<CapacityLimiter.Ticket> onCancellation)

Ticket terminal callback override upon cancellation of the request operation. By default the terminal callback is CapacityLimiter.Ticket.dropped().

You may need to adjust this callback depending on the ordering this filter was applied. For example if the filter is applied after the timeout-filter then you may want to also drop the ticket to let the algorithm apply throttling accounting for this timeout.

Parameters:

onCancellation - Callback to override default ticket terminal event when an operation is cancelled.

Returns:

this.

observer

public TrafficResilienceHttpClientFilter.Builder observer(TrafficResiliencyObserver observer)

Provide an observer to track interactions of the filter and requests.

Parameters:

observer - an observer to track interactions of the filter and requests.

Returns:

this.

dryRun

public TrafficResilienceHttpClientFilter.Builder dryRun(boolean dryRun)

Use the resilience filter in dry-run mode. In dry-run mode the capacity limiter will track requests and log their results but request which would have been rejected will instead pass through to the underlying client.

Parameters:

dryRun - whether to use the resilience filter in dry-run mode.

Returns:

this

build

public TrafficResilienceHttpClientFilter build()

Invoke to build an instance of TrafficResilienceHttpClientFilter filter to be used inside the HttpClientBuilder.

Returns:

An instance of TrafficResilienceHttpClientFilter with the characteristics of this builder input.