io.servicetalk.traffic.resilience.http

Class TrafficResilienceHttpServiceFilter.Builder

Object

TrafficResilienceHttpServiceFilter.Builder

Enclosing class:

TrafficResilienceHttpServiceFilter

public static final class TrafficResilienceHttpServiceFilter.Builder
extends Object

A TrafficResilienceHttpServiceFilter instance builder.

Constructor Summary

Constructors

Constructor and Description
Builder(Supplier<CapacityLimiter> capacityLimiterSupplier) A TrafficResilienceHttpServiceFilter with no partitioning schemes.
Builder(Supplier<Function<HttpRequestMetaData,CapacityLimiter>> capacityPartitionsSupplier, boolean rejectNotMatched) A TrafficResilienceHttpServiceFilter can support request partitioning schemes.

Method Summary

Modifier and Type	Method and Description
TrafficResilienceHttpServiceFilter	build() Invoke to build an instance of TrafficResilienceHttpServiceFilter filter to be used inside the HttpServerBuilder.
TrafficResilienceHttpServiceFilter.Builder	capacityPartitions(Supplier<Function<HttpRequestMetaData,CapacityLimiter>> capacityPartitionsSupplier, boolean rejectNotMatched) Define CapacityLimiter partitions.
TrafficResilienceHttpServiceFilter.Builder	circuitBreakerPartitions(Supplier<Function<HttpRequestMetaData,CircuitBreaker>> circuitBreakerPartitionsSupplier) Define CircuitBreaker partitions to manage local errors.
TrafficResilienceHttpServiceFilter.Builder	classifier(Supplier<Function<HttpRequestMetaData,Classification>> classifier) Classification in the context of capacity management allows for hints to the relevant CapacityLimiter to be influenced on the decision making process by the class of the request.
TrafficResilienceHttpServiceFilter.Builder	dryRun(boolean dryRun) Use the resilience filter in dry-run mode.
TrafficResilienceHttpServiceFilter.Builder	observer(TrafficResiliencyObserver observer) Provide an observer to track interactions of the filter and requests.
TrafficResilienceHttpServiceFilter.Builder	onCancelTicketTerminal(Consumer<CapacityLimiter.Ticket> onCancellation) Ticket terminal callback override upon cancellation of the operation.
TrafficResilienceHttpServiceFilter.Builder	onErrorTicketTerminal(BiConsumer<CapacityLimiter.Ticket,Throwable> onError) Ticket terminal callback override upon erroneous completion of the operation.
TrafficResilienceHttpServiceFilter.Builder	rejectionPolicy(ServiceRejectionPolicy policy) Defines the ServiceRejectionPolicy which in turn defines the behavior of the service when a rejection occurs due to capacity or breaker.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Builder

public Builder(Supplier<CapacityLimiter> capacityLimiterSupplier)

A TrafficResilienceHttpServiceFilter with no partitioning schemes.

All requests will go through the provided CapacityLimiter.

Parameters:

capacityLimiterSupplier - The Supplier to create a new CapacityLimiter for each new filter created by this factory.

Builder

public Builder(Supplier<Function<HttpRequestMetaData,CapacityLimiter>> capacityPartitionsSupplier,
               boolean rejectNotMatched)

A TrafficResilienceHttpServiceFilter can support request partitioning schemes.

A partition in the context of capacity management, is a set of requests that represent an application characteristic relevant to capacity, which can be isolated and have their own set of rules (ie. CapacityLimiter).

An example of a partition can be to represent each customer in a multi-tenant service. If an application wants to introduce customer API quotas, they can do so by identifying that customer through the HttpRequestMetaData and providing a different CapacityLimiter for that customer.

If a partitions doesn't return a CapacityLimiter for the given HttpRequestMetaData then the rejectNotMatched is evaluated to decide what the filter should do with this request. If true then the request will be rejected.

It's important that instances returned from this mapper are singletons and shared across the same matched partitions. Otherwise, capacity will not be controlled as expected, and there is also the risk for OutOfMemoryError.

Parameters:

capacityPartitionsSupplier - A Supplier to create a new Function for each new filter created by this factory. Function provides a CapacityLimiter instance for the given HttpRequestMetaData.

rejectNotMatched - Flag that decides what the filter should do when partitions doesn't return a CapacityLimiter.

Method Detail

capacityPartitions

public TrafficResilienceHttpServiceFilter.Builder capacityPartitions(Supplier<Function<HttpRequestMetaData,CapacityLimiter>> capacityPartitionsSupplier,
                                                                     boolean rejectNotMatched)

Define CapacityLimiter partitions.

A partition in the context of capacity management, is a set of requests that represent an application characteristic relevant to capacity, which can be isolated and have their own set of rules (i.e. CapacityLimiter).

An example of a partition can be to represent each customer in a multi-tenant service. If an application wants to introduce customer API quotas, they can do so by identifying that customer through the HttpRequestMetaData and providing a different CapacityLimiter for that customer.

If a partitions doesn't return a CapacityLimiter for the given HttpRequestMetaData then the rejectNotMatched is evaluated to decide what the filter should do with this request. If true then the request will be rejected.

It's important that instances returned from this mapper are singletons and shared across the same matched partitions. Otherwise, capacity will not be controlled as expected, and there is also the risk for OutOfMemoryError.

Parameters:

capacityPartitionsSupplier - A Supplier to create a new Function for each new filter created by this factory. Function provides a CapacityLimiter instance for the given HttpRequestMetaData.

rejectNotMatched - Flag that decides what the filter should do when partitions doesn't return a CapacityLimiter.

Returns:

this.

classifier

public TrafficResilienceHttpServiceFilter.Builder classifier(Supplier<Function<HttpRequestMetaData,Classification>> classifier)

Classification in the context of capacity management allows for hints to the relevant CapacityLimiter to be influenced on the decision making process by the class of the request.

An example of classification, could be health checks that need to be given preference and still allowed a permit even under stress conditions. Another case, could be a separation of reads and writes, giving preference to the reads will result in a more available system under stress, by rejecting earlier writes.

The behavior of the classification and their thresholds could be different among different CapacityLimiter implementations, therefore the use of this API requires good understanding of how the algorithm in use will react for the different classifications.

Classification work within the context of a single Builder(Supplier, boolean) partition} and not universally in the filter.

Parameters:

classifier - A Supplier of a Function that maps an incoming HttpRequestMetaData to a Classification.

Returns:

this.

circuitBreakerPartitions

public TrafficResilienceHttpServiceFilter.Builder circuitBreakerPartitions(Supplier<Function<HttpRequestMetaData,CircuitBreaker>> circuitBreakerPartitionsSupplier)

Define CircuitBreaker partitions to manage local errors.

The breakers can either be universal or follow any partitioning scheme (i.e., API / service-path, customer e.t.c) but is recommended to follow similar schematics between service and client if possible for best experience.

Once a matching CircuitBreaker transitions to open state, requests that match the same breaker will fail (e.g., HttpResponseStatus.SERVICE_UNAVAILABLE) and ServiceRejectionPolicy.onOpenCircuitRetryAfter() can be used to hint peers about the fact that the circuit will remain open for a certain amount of time.

Parameters:

circuitBreakerPartitionsSupplier - A Supplier to create a new Function for each new filter created by this factory. Function provides a CircuitBreaker instance for the given HttpRequestMetaData.

Returns:

this.

onErrorTicketTerminal

public TrafficResilienceHttpServiceFilter.Builder onErrorTicketTerminal(BiConsumer<CapacityLimiter.Ticket,Throwable> onError)

Ticket terminal callback override upon erroneous completion of the operation. Erroneous completion in this context means, that an error occurred for either the Single or the Publisher of the operation. By default, the terminal callback is CapacityLimiter.Ticket.failed(Throwable).

Parameters:

onError - Callback to override default ticket terminal event for an erroneous operation.

Returns:

this.

onCancelTicketTerminal

public TrafficResilienceHttpServiceFilter.Builder onCancelTicketTerminal(Consumer<CapacityLimiter.Ticket> onCancellation)

Ticket terminal callback override upon cancellation of the operation. By default, the terminal callback is CapacityLimiter.Ticket.ignored().

You may need to adjust this callback depending on the ordering this filter was applied. For example if the filter is applied after the timeout-filter then you may want to also drop the ticket to let the algorithm apply throttling accounting for this timeout.

Parameters:

onCancellation - Callback to override default ticket terminal event when an operation is cancelled.

Returns:

this.

rejectionPolicy

public TrafficResilienceHttpServiceFilter.Builder rejectionPolicy(ServiceRejectionPolicy policy)

Defines the ServiceRejectionPolicy which in turn defines the behavior of the service when a rejection occurs due to capacity or breaker.

Parameters:

policy - The policy to put into effect when a rejection occurs.

Returns:

this.

See Also:

ServiceRejectionPolicy.DEFAULT_REJECTION_POLICY

observer

public TrafficResilienceHttpServiceFilter.Builder observer(TrafficResiliencyObserver observer)

Provide an observer to track interactions of the filter and requests.

Parameters:

observer - an observer to track interactions of the filter and requests.

Returns:

this.

dryRun

public TrafficResilienceHttpServiceFilter.Builder dryRun(boolean dryRun)

Use the resilience filter in dry-run mode. In dry-run mode the capacity limiter will track requests and log their results but request which would have been rejected will instead pass through to the underlying client.

Parameters:

dryRun - whether to use the resilience filter in dry-run mode.

Returns:

this

build

public TrafficResilienceHttpServiceFilter build()

Invoke to build an instance of TrafficResilienceHttpServiceFilter filter to be used inside the HttpServerBuilder.

Returns:

An instance of TrafficResilienceHttpServiceFilter with the characteristics of this builder input.