org.apache.spark.sql.delta.sources
The version of serialization that this offset is encoded with.
The id of the table we are reading from. Used to detect misconfiguration when restarting a query.
The version of the table that we are current processing.
The index in the sequence of AddFiles in this version. Used to break large commits into multiple batches. This index is created by sorting on modificationTimestamp and path.
Whether this offset denotes a query that is starting rather than processing changes. When starting a new query, we first process all data present in the table at the start and then move on to processing new data that has arrived.
The index in the sequence of AddFiles in this version.
The index in the sequence of AddFiles in this version. Used to break large commits into multiple batches. This index is created by sorting on modificationTimestamp and path.
Whether this offset denotes a query that is starting rather than processing changes.
Whether this offset denotes a query that is starting rather than processing changes. When starting a new query, we first process all data present in the table at the start and then move on to processing new data that has arrived.
The id of the table we are reading from.
The id of the table we are reading from. Used to detect misconfiguration when restarting a query.
The version of the table that we are current processing.
The version of serialization that this offset is encoded with.
Tracks how far we processed in when reading changes from the DeltaLog.
Note this class retains the naming of
Reservoir
to maintain compatibility with serialized offsets from the beta period.The version of serialization that this offset is encoded with.
The id of the table we are reading from. Used to detect misconfiguration when restarting a query.
The version of the table that we are current processing.
The index in the sequence of AddFiles in this version. Used to break large commits into multiple batches. This index is created by sorting on modificationTimestamp and path.
Whether this offset denotes a query that is starting rather than processing changes. When starting a new query, we first process all data present in the table at the start and then move on to processing new data that has arrived.