Builder

com.github.mjakubowski84.parquet4s.parquet.reader.Builder
trait Builder[F[_], T]

Attributes

Graph
Supertypes
class Object
trait Matchable
class Any

Members list

Value members

Abstract methods

def chunkSize(chunkSize: Int): Builder[F, T]

For sake of better performance reader processes records in chunks. Default value is 16.

For sake of better performance reader processes records in chunks. Default value is 16.

Value parameters

chunkSize

default value override

Attributes

def filter(filter: Filter): Builder[F, T]

Value parameters

filter

optional before-read filter; no filtering is applied by default; check Filter for more details

Attributes

def options(options: Options): Builder[F, T]

Value parameters

options

configuration of how Parquet files should be read

Attributes

def parallelism(n: Int)(implicit concurrent: Concurrent[F]): Builder[F, T]

How many files at most shall be read in parallel. Default value is 1.

How many files at most shall be read in parallel. Default value is 1.

Value parameters

concurrent

parallel reading requires concurrency

n

desired parallelism

Attributes

def read(path: Path): Stream[F, T]

Value parameters

path

Path to Parquet files, e.g.:

Path("file:///data/users")

Attributes

Returns

final fs2.Stream

def read(inputFile: InputFile): Stream[F, T]

Value parameters

inputFile

file to read

Attributes

Returns

final fs2.Stream

def readFileStreams(path: Path): Stream[F, Stream[F, T]]

Value parameters

path

Path to Parquet files, e.g.:

Path("file:///data/users")

Attributes

Returns

fs2.Stream of streams of content of each individual Parquet file

def readFileStreams(inputFile: InputFile): Stream[F, Stream[F, T]]

Value parameters

inputFile

file to read

Attributes

Returns

fs2.Stream of streams of content of each individual Parquet file