com.linkedin.parseq.batching

Class TaskBatchingStrategy<G,K,T>

java.lang.Object

com.linkedin.parseq.batching.BatchingStrategy<G,K,T>

com.linkedin.parseq.batching.TaskBatchingStrategy<G,K,T>

Type Parameters:

G - Type of a Group

K - Type of a Key

T - Type of a Value

Direct Known Subclasses:

SimpleTaskBatchingStrategy

public abstract class TaskBatchingStrategy<G,K,T>
extends BatchingStrategy<G,K,T>

This is a base class for a batching strategy that leverages existing Task-based API.

Example below shows how to build a ParSeq client for a key-value store that provides transparent batching given existing Task-based API. Let's assume that we have an implementation of the following key-value store interface:

  interface KVStore {
    Task <String> get(Long key);
    Task <Map<Long, Try<String>>> batchGet(Collection <Long> keys);
  }
 

We can then implement a TaskBatchingStrategy in the following way (for the sake of simplicity we assume that all keys can be grouped into one batch thus we implement SimpleTaskBatchingStrategy):

  public static class BatchingKVStoreClient extends SimpleTaskBatchingStrategy <Long, String> {
    private final KVStore _store;
    public BatchingKVStoreClient(KVStore store) {
      _store = store;
    }

     @Override
    public void executeBatch(Integer group, Batch <Long, String> batch) {
      Map <Long, String> batchResult = _store.batchGet(batch.keys());
      batch.foreach((key, promise)  -> promise.done(batchResult.get(key)));
    }

     @Override
    public  Task<Map<Long, Try<String>>> taskForBatch(Set <Long> keys) {
      return _store.batchGet(keys);
    }
  }
 

taskForBatch method returns a task that computes a map that for every key contains either a success with a value or a failure. If returned map does not contain results for some keys the tasks for which results are missing will fail.

Author:

Jaroslaw Odzga (jodzga@linkedin.com)

See Also:

BatchingStrategy, SimpleTaskBatchingStrategy

Field Summary

Fields inherited from class com.linkedin.parseq.batching.BatchingStrategy

DEFAULT_MAX_BATCH_SIZE

Constructor Summary

Constructors

Constructor and Description
TaskBatchingStrategy()

Method Summary

Modifier and Type	Method and Description
void	executeBatch(G group, Batch<K,T> batch) This method will be called for every Batch.
protected void	executeBatchWithContext(G group, Batch<K,T> batch, com.linkedin.parseq.Context ctx)
String	getBatchName(G group, Batch<K,T> batch) Overriding this method allows providing custom name for a batch.
String	getBatchName(G group, Set<K> keys) Overriding this method allows providing custom name for a batch.
abstract com.linkedin.parseq.Task<Map<K,com.linkedin.parseq.function.Try<T>>>	taskForBatch(G group, Set<K> keys) This method will be called for every batch.

Methods inherited from class com.linkedin.parseq.batching.BatchingStrategy

batchable, batchable, classify, getBatchAggregationTimeMetric, getBatchSizeMetric, keySize, maxBatchSizeForGroup

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TaskBatchingStrategy

public TaskBatchingStrategy()

Method Detail

executeBatch

public final void executeBatch(G group,
                               Batch<K,T> batch)

Description copied from class: BatchingStrategy

This method will be called for every Batch. Implementation of this method must make sure that all SettablePromise contained in the Batch will eventually be resolved - typically asynchronously. Failing to eventually resolve any of the promises may lead to plan that never completes i.e. appears to hung and may lead to a memory leak.

Specified by:

executeBatch in class BatchingStrategy<G,K,T>

Parameters:

group - group that represents the batch

batch - batch contains collection of SettablePromise that eventually need to be resolved - typically asynchronously

executeBatchWithContext

protected void executeBatchWithContext(G group,
                                       Batch<K,T> batch,
                                       com.linkedin.parseq.Context ctx)

Overrides:

executeBatchWithContext in class BatchingStrategy<G,K,T>

getBatchName

public final String getBatchName(G group,
                                 Batch<K,T> batch)

Description copied from class: BatchingStrategy

Overriding this method allows providing custom name for a batch. Name will appear in the ParSeq trace as a description of the task that executes the batch.

Overrides:

getBatchName in class BatchingStrategy<G,K,T>

Parameters:

group - group to be described

batch - batch to be described

Returns:

name for the batch and group

getBatchName

public String getBatchName(G group,
                           Set<K> keys)

Overriding this method allows providing custom name for a batch. Name will appear in the ParSeq trace as a description of the task that executes the batch.

Parameters:

keys - set of keys belonging to the batch that needs to be described

group - group to be described

Returns:

name for the batch

taskForBatch

public abstract com.linkedin.parseq.Task<Map<K,com.linkedin.parseq.function.Try<T>>> taskForBatch(G group,
                                                                                                  Set<K> keys)

This method will be called for every batch. It returns a map that for every key contains either a success with a value or a failure. If returned map does not contain results for some keys the tasks for which results are missing will fail.

Parameters:

group - group that represents the batch

keys - set of keys belonging to the batch

Returns:

A map that for every key contains either a success with a value or a failure. If returned map does not contain results for some keys the tasks for which results are missing will fail.