ParquetPartitioningFlow

Type parameters

T

Type of message that flow accepts

W

Schema of Parquet file that flow writes

Attributes

Supertypes

class Object

trait Matchable

class Any

Known subtypes

trait GenericBuilder

trait TypedBuilder[T, W]

Attributes

Supertypes

trait Builder[RowParquetRecord, RowParquetRecord, GenericBuilder]

class Object

trait Matchable

class Any

Type parameters

T

type of input data

Value parameters

flush

Flushes all writes to given partition and rotates the file.

modifiedPartitions

State of partitions that has been written in effect of processing the element T. More than one partition can be modified due to preWriteTransformation. The map contains values representing total number of writes to a single file (number of writes to the partition after last rotation).

processedData

Processed input element

Attributes

Supertypes

trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any

Show all

Attributes

Supertypes

trait Builder[T, W, TypedBuilder[T, W]]

class Object

trait Matchable

class Any

Attributes

Supertypes

class Object

trait Matchable

class Any