package cloud
- Alphabetic
- Public
- Protected
Type Members
- class BindingParquetOutputCommitter extends ParquetOutputCommitter with Logging
This Parquet Committer subclass dynamically binds to the factory-configured output committer, and is intended to allow callers to use any 'PathOutputCommitter', even if not a subclass of 'ParquetOutputCommitter'.
This Parquet Committer subclass dynamically binds to the factory-configured output committer, and is intended to allow callers to use any 'PathOutputCommitter', even if not a subclass of 'ParquetOutputCommitter'.
The Parquet
parquet.enable.summary-metadataoption will only be supported if the instantiated committer itself supports it.Copied without changes from https://github.com/apache/spark/tree/master/hadoop-cloud/src/hadoop-3/main/scala/org/apache/spark/internal/io/cloud as there is no release from org.apache.spark:hadoop-cloud
- class PathOutputCommitProtocol extends HadoopMapReduceCommitProtocol with Serializable
Spark Commit protocol for Path Output Committers.
Spark Commit protocol for Path Output Committers. This committer will work with the
FileOutputCommitterand subclasses. All implementations *must* be serializable.Rather than ask the
FileOutputFormatfor a committer, it uses theorg.apache.hadoop.mapreduce.lib.output.PathOutputCommitterFactoryfactory API to create the committer.In
setupCommitterthe factory is identified and instantiated; this factory then creates the actual committer implementation.Copied without changes from https://github.com/apache/spark/tree/master/hadoop-cloud/src/hadoop-3/main/scala/org/apache/spark/internal/io/cloud as there is no release from org.apache.spark:hadoop-cloud *
Value Members
- object PathOutputCommitProtocol extends Serializable