Reads the latest partition of a given table.
Reads the latest partition of a given table.
In order to read a table it is not sufficient the table to be registered in the metastore. It also should be defined as input tables of the job. Otherwise, a runtime exception will be thrown.
The name of the table to read.
An optional upper boundary. When you run historical transformations you might want to limit the recency of input data. Uses the current information date if None.
The dataframe containing data from the table.
Returns the latest information date the table has data for.
Returns the latest information date the table has data for.
In order to read a table it is not sufficient the table to be registered in the metastore. It also should be defined as input tables of the job. Otherwise, a runtime exception will be thrown.
The name of the table to read.
An optional upper boundary. When you run historical transformations you might want to limit the recency of input data.
The latest information date the table has data for, None otherwise.
Reads a table given th range of information dates, and returns back the dataframe.
Reads a table given th range of information dates, and returns back the dataframe.
In order to read a table it is not sufficient the table to be registered in the metastore. It also should be defined as input tables of the job. Otherwise, a runtime exception will be thrown.
The name of the table to read.
The starting info date to fetch data from (inclusive). Uses the current information date if None.
The ending info date (inclusive). Uses the current information date if None.
The dataframe containing data from the table.
Returns true if data for the specified table is available for the specified range.
Returns true if data for the specified table is available for the specified range.
This method can be used for validations.
The name of the table to read.
The starting info date of the availability of the table (inclusive).
An upper boundary. When you run historical transformations you might want to limit the recency of input data.
true if data is available for the specified range.
Metastore reader allows querying tables registered at the 'metastore' section of the configuration. It abstracts away the storage provider (HDFS, S3, etc), format (Parquet, Delta, etc.) and partitioning options.