A job has access to the metastore where it can query tables registered in the configuration.
A job has access to the metastore where it can query tables registered in the configuration.
If task fails it should throw a runtime exception.
Do not add information date column to the output dataframe. It will be added automatically. If the information date column exists in the output dataframe, it will be deleted and replaced by the framework.
The read only version of metastore. You can only query tables using it.
The information date of the output of the transformation.
Extra options specified in the configuration for the transformation.
The output DataFrame
Validates if the transformation can run.
Validates if the transformation can run.
If the transformation can run for the given information date, it should return Reason.Ready.
If requirements are not met the validation routine should do one of teh following: - return Reason.NotReady - this means some input data is missing, but the transformation can run in the future when the requirements are met - return Reason.Skip - this means the running of the transformation should be completely skipped. - throw an exception - this will result in the same behavior as returning Reason.NotReady
The read only version of metastore. You can only query tables using it.
The information date of the output of the transformation.
Extra options specified in the configuration for the transformation.
Reason as the validation result.