Write the dataframe to files.
Write the dataframe to files. This gets called in the driver by the FileStreamSink.
Writes a RDD partition to multiple dynamically partitioned files.
Writes a RDD partition to multiple dynamically partitioned files. This gets called in the executor. It first sorts the data based on the partitioning columns and then writes the data of each key to separate files using OutputWriters.
Writes a RDD partition to a single file without dynamic partitioning.
Writes a RDD partition to a single file without dynamic partitioning. This gets called in the executor, and it uses a OutputWriter to write the data.
Writes data given to a FileStreamSink to the given
basePath
in the givenfileFormat
, partitioned by the givenpartitionColumnNames
. This writer always appends data to the directory if it already has data.