A B C D E F G I L P R S T W

A

absoluteFilePath() - Method in class io.delta.storage.ExternalCommitEntry

absoluteTempPath() - Method in class io.delta.storage.ExternalCommitEntry

asComplete() - Method in class io.delta.storage.ExternalCommitEntry

B

BASE_CONF_PREFIX - Static variable in class io.delta.storage.S3DynamoDBLogStore

BaseExternalLogStore - Class in io.delta.storage

A base LogStore implementation for cloud stores (e.g.

BaseExternalLogStore(Configuration) - Constructor for class io.delta.storage.BaseExternalLogStore

C

commitTime - Variable in class io.delta.storage.ExternalCommitEntry

epoch seconds of time of commit if complete=true, else null

complete - Variable in class io.delta.storage.ExternalCommitEntry

true if delta json file is successfully copied to its destination location, else false

createAwsCredentialsProvider(String, Configuration) - Static method in class io.delta.storage.utils.ReflectionUtils

Create AWS credentials provider from given provider classname and Configuration.

createTemporaryPath(Path) - Method in class io.delta.storage.BaseExternalLogStore

Generate temporary path for TransactionLog.

D

DDB_CLIENT_CREDENTIALS_PROVIDER - Static variable in class io.delta.storage.S3DynamoDBLogStore

DDB_CLIENT_REGION - Static variable in class io.delta.storage.S3DynamoDBLogStore

DDB_CLIENT_TABLE - Static variable in class io.delta.storage.S3DynamoDBLogStore

DDB_CREATE_TABLE_RCU - Static variable in class io.delta.storage.S3DynamoDBLogStore

DDB_CREATE_TABLE_WCU - Static variable in class io.delta.storage.S3DynamoDBLogStore

E

ExternalCommitEntry - Class in io.delta.storage

Wrapper class representing an entry in an external store for a given commit into the Delta log.

ExternalCommitEntry(Path, String, String, boolean, Long) - Constructor for class io.delta.storage.ExternalCommitEntry

F

fileName - Variable in class io.delta.storage.ExternalCommitEntry

File name of this commit, e.g.

fixDeltaLogCopyTempFile(FileSystem, Path, Path) - Method in class io.delta.storage.BaseExternalLogStore

Wrapper for `copyFile`, called by the `fixDeltaLog` method.

fixDeltaLogPutCompleteDbEntry(ExternalCommitEntry) - Method in class io.delta.storage.BaseExternalLogStore

Wrapper for `putExternalEntry`, called by the `fixDeltaLog` method.

G

getExternalEntry(String, String) - Method in class io.delta.storage.BaseExternalLogStore

Return external store entry corresponding to delta log file with given `tablePath` and `fileName`, or `Optional.empty()` if it doesn't exist.

getExternalEntry(String, String) - Method in class io.delta.storage.S3DynamoDBLogStore

getLatestExternalEntry(Path) - Method in class io.delta.storage.BaseExternalLogStore

Return the latest external store entry corresponding to the delta log for given `tablePath`, or `Optional.empty()` if it doesn't exist.

getLatestExternalEntry(Path) - Method in class io.delta.storage.S3DynamoDBLogStore

getParam(Configuration, String, String) - Static method in class io.delta.storage.S3DynamoDBLogStore

Get the hadoopConf param $name that is prefixed either with $SPARK_CONF_PREFIX or $BASE_CONF_PREFIX.

getTablePath(Path) - Method in class io.delta.storage.BaseExternalLogStore

Returns the base table path for a given Delta log entry located in e.g.

I

io.delta.storage - package io.delta.storage

io.delta.storage.utils - package io.delta.storage.utils

isPartialWriteVisible(Path, Configuration) - Method in class io.delta.storage.BaseExternalLogStore

L

listFrom(Path, Configuration) - Method in class io.delta.storage.BaseExternalLogStore

First checks if there is any incomplete entry in the external store.

P

putExternalEntry(ExternalCommitEntry, boolean) - Method in class io.delta.storage.BaseExternalLogStore

Write to external store in exclusive way.

putExternalEntry(ExternalCommitEntry, boolean) - Method in class io.delta.storage.S3DynamoDBLogStore

R

ReflectionUtils - Class in io.delta.storage.utils

ReflectionUtils() - Constructor for class io.delta.storage.utils.ReflectionUtils

S

S3DynamoDBLogStore - Class in io.delta.storage

A concrete implementation of BaseExternalLogStore that uses an external DynamoDB table to provide the mutual exclusion during calls to `putExternalEntry`.

S3DynamoDBLogStore(Configuration) - Constructor for class io.delta.storage.S3DynamoDBLogStore

SPARK_CONF_PREFIX - Static variable in class io.delta.storage.S3DynamoDBLogStore

Configuration keys for the DynamoDB client.

T

tablePath - Variable in class io.delta.storage.ExternalCommitEntry

Absolute path to this delta table

tempPath - Variable in class io.delta.storage.ExternalCommitEntry

Path to temp file for this commit, relative to the `_delta_log

W

write(Path, Iterator<String>, Boolean, Configuration) - Method in class io.delta.storage.BaseExternalLogStore

If overwrite=true, then write normally without any interaction with external store.

writeActions(FileSystem, Path, Iterator<String>) - Method in class io.delta.storage.BaseExternalLogStore

Write file with actions under a specific path.

writeCopyTempFile(FileSystem, Path, Path) - Method in class io.delta.storage.BaseExternalLogStore

Wrapper for `copyFile`, called by the `write` method.

writePutCompleteDbEntry(ExternalCommitEntry) - Method in class io.delta.storage.BaseExternalLogStore

Wrapper for `putExternalEntry`, called by the `write` method.

A B C D E F G I L P R S T W