io.druid.segment

Interface DimensionHandler<EncodedType extends Comparable<EncodedType>,EncodedTypeArray,ActualType extends Comparable<ActualType>>

Type Parameters:

EncodedType - class of the encoded values

ActualType - class of the actual values

All Known Implementing Classes:

StringDimensionHandler

public interface DimensionHandler<EncodedType extends Comparable<EncodedType>,EncodedTypeArray,ActualType extends Comparable<ActualType>>

Processing related interface A DimensionHandler is an object that encapsulates indexing, column merging/building, and querying operations for a given dimension type (e.g., dict-encoded String, Long). These operations are handled by sub-objects created through a DimensionHandler's methods: DimensionIndexer, DimensionMerger, and DimensionColumnReader, respectively. Each DimensionHandler object is associated with a single dimension. This interface allows type-specific behavior column logic, such as choice of indexing structures and disk formats. to be contained within a type-specific set of handler objects, simplifying processing classes such as IncrementalIndex and IndexMerger and allowing for abstracted development of additional dimension types. A dimension may have two representations, an encoded representation and a actual representation. For example, a value for a String dimension has an integer dictionary encoding, and an actual String representation. A DimensionHandler is a stateless object, and thus thread-safe; its methods should be pure functions. The EncodedType and ActualType are Comparable because columns used as dimensions must have sortable values.

Method Summary

Modifier and Type	Method and Description
int	compareSortedEncodedArrays(EncodedTypeArray lhs, EncodedTypeArray rhs) Given two arrays representing sorted encoded row value(s), return the result of their comparison.
String	getDimensionName() Get the name of the column associated with this handler.
int	getLengthFromEncodedArray(EncodedTypeArray dimVals) Given an array representing a single set of row value(s) for this dimension as an Object, return the length of the array after appropriate type-casting.
Object	getRowValueArrayFromColumn(Closeable column, int currRow) Given a subcolumn from getSubColumn, and the index of the current row, retrieve a row as an array of values.
Closeable	getSubColumn(Column column) Given a Column, return a type-specific object that can be used to retrieve row values.
DimensionIndexer<EncodedType,EncodedTypeArray,ActualType>	makeIndexer() Creates a new DimensionIndexer, a per-dimension object responsible for processing ingested rows in-memory, used by the IncrementalIndex.
DimensionMergerLegacy<EncodedTypeArray>	makeLegacyMerger(IndexSpec indexSpec, File outDir, IOPeon ioPeon, ColumnCapabilities capabilities, ProgressIndicator progress) Creates a new DimensionMergerLegacy, a per-dimension object responsible for merging indexes/row data across segments and building the on-disk representation of a dimension.
DimensionMergerV9<EncodedTypeArray>	makeMerger(IndexSpec indexSpec, File outDir, IOPeon ioPeon, ColumnCapabilities capabilities, ProgressIndicator progress) Creates a new DimensionMergerV9, a per-dimension object responsible for merging indexes/row data across segments and building the on-disk representation of a dimension.
void	validateSortedEncodedArrays(EncodedTypeArray lhs, EncodedTypeArray rhs, Indexed<ActualType> lhsEncodings, Indexed<ActualType> rhsEncodings) Given two arrays representing sorted encoded row value(s), check that the two arrays have the same encoded values, or if the encoded values differ, that they translate into the same actual values, using the mappings provided by lhsEncodings and rhsEncodings (if applicable).

Method Detail

getDimensionName

String getDimensionName()

Get the name of the column associated with this handler. This string would be the output name of the column during ingestion, and the name of an input column when querying.

Returns:

Dimension name

makeIndexer

DimensionIndexer<EncodedType,EncodedTypeArray,ActualType> makeIndexer()

Creates a new DimensionIndexer, a per-dimension object responsible for processing ingested rows in-memory, used by the IncrementalIndex. See DimensionIndexer interface for more information.

Returns:

A new DimensionIndexer object.

makeMerger

DimensionMergerV9<EncodedTypeArray> makeMerger(IndexSpec indexSpec,
                                               File outDir,
                                               IOPeon ioPeon,
                                               ColumnCapabilities capabilities,
                                               ProgressIndicator progress)

Creates a new DimensionMergerV9, a per-dimension object responsible for merging indexes/row data across segments and building the on-disk representation of a dimension. For use with IndexMergerV9 only. See DimensionMergerV9 interface for more information.

Parameters:

indexSpec - Specification object for the index merge

outDir - Location to store files generated by the merging process

ioPeon - ioPeon object passed in by IndexMerger, manages files created by the merging process

capabilities - The ColumnCapabilities of the dimension represented by this DimensionHandler

progress - ProgressIndicator used by the merging process

Returns:

A new DimensionMergerV9 object.

makeLegacyMerger

DimensionMergerLegacy<EncodedTypeArray> makeLegacyMerger(IndexSpec indexSpec,
                                                         File outDir,
                                                         IOPeon ioPeon,
                                                         ColumnCapabilities capabilities,
                                                         ProgressIndicator progress)

Creates a new DimensionMergerLegacy, a per-dimension object responsible for merging indexes/row data across segments and building the on-disk representation of a dimension. For use with IndexMerger only. See DimensionMergerLegacy interface for more information.

Parameters:

indexSpec - Specification object for the index merge

outDir - Location to store files generated by the merging process

ioPeon - ioPeon object passed in by IndexMerger, manages files created by the merging process

capabilities - The ColumnCapabilities of the dimension represented by this DimensionHandler

progress - ProgressIndicator used by the merging process

Returns:

A new DimensionMergerLegacy object.

getLengthFromEncodedArray

int getLengthFromEncodedArray(EncodedTypeArray dimVals)

Given an array representing a single set of row value(s) for this dimension as an Object, return the length of the array after appropriate type-casting. For example, a dictionary encoded String dimension would receive an int[] as an Object.

Parameters:

dimVals - Array of row values

Returns:

Size of dimVals

compareSortedEncodedArrays

int compareSortedEncodedArrays(EncodedTypeArray lhs,
                               EncodedTypeArray rhs)

Given two arrays representing sorted encoded row value(s), return the result of their comparison. If the two arrays have different lengths, the shorter array should be ordered first in the comparison. Otherwise, this function should iterate through the array values and return the comparison of the first difference.

Parameters:

lhs - array of row values

rhs - array of row values

Returns:

integer indicating comparison result of arrays

validateSortedEncodedArrays

void validateSortedEncodedArrays(EncodedTypeArray lhs,
                                 EncodedTypeArray rhs,
                                 Indexed<ActualType> lhsEncodings,
                                 Indexed<ActualType> rhsEncodings)
                          throws SegmentValidationException

Given two arrays representing sorted encoded row value(s), check that the two arrays have the same encoded values, or if the encoded values differ, that they translate into the same actual values, using the mappings provided by lhsEncodings and rhsEncodings (if applicable). If validation fails, this method should throw a SegmentValidationException. Used by IndexIO for validating segments. See StringDimensionHandler.validateSortedEncodedArrays() for a reference implementation.

Parameters:

lhs - array of row values

rhs - array of row values

lhsEncodings - encoding lookup from lhs's segment, null if not applicable for this dimension's type

rhsEncodings - encoding lookup from rhs's segment, null if not applicable for this dimension's type

Throws:

SegmentValidationException

getSubColumn

Closeable getSubColumn(Column column)

Given a Column, return a type-specific object that can be used to retrieve row values. For example: - A String-typed implementation would return the result of column.getDictionaryEncoding() - A long-typed implemention would return the result of column.getGenericColumn().

Parameters:

column - Column for this dimension from a QueryableIndex

Returns:

The type-specific column subobject for this dimension.

getRowValueArrayFromColumn

Object getRowValueArrayFromColumn(Closeable column,
                                  int currRow)

Given a subcolumn from getSubColumn, and the index of the current row, retrieve a row as an array of values. For example: - A String-typed implementation would read the current row from a DictionaryEncodedColumn as an int[]. - A long-typed implemention would read the current row from a GenericColumn return the current row as a long[].

Parameters:

column - Column for this dimension from a QueryableIndex

currRow - The index of the row to retrieve

Returns:

The row from "column" specified by "currRow", as an array of values