com.coxautodata.waimak.dataflow.spark
Implements a cleanup strategy that sorts input list of snapshots by timestamp extracted from the folder names and ensures that there is at most numberOfFoldersToKeep of folders with latest timestamp left.
Implements a cleanup strategy that sorts input list of snapshots by timestamp extracted from the folder names and ensures that there is at most numberOfFoldersToKeep of folders with latest timestamp left. Folder names must be of same pattern as hive partition columns. Example: COLUMNNAME=TIMESTAMP
type that identifies snapshot
column name part of the snapshot folder
Java format of the TIMESTAMP. Ex: yyyyMMddHHmmss
maximum number of snapshots to keep
returns name of the snapshot
configured cleanup strategy that returns list of snapshots to remove