HDFSCheckpointStore

Value Members

1. final def !=(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

2. final def ##(): Int

   

   Definition Classes

   AnyRef → Any

3. final def ==(arg0: Any): Boolean

   

   Definition Classes

   AnyRef → Any

4. final def asInstanceOf[T0]: T0

   

   Definition Classes

   Any

5. implicit val batcher: Batcher

   

   Definition Classes

   HDFSCheckpointStore → CheckpointStore

6. def checkpointBatchStart(intersection: Intersection[InclusiveLower, ExclusiveUpper, Timestamp]): Iterable[BatchID]

   

   is called when the batch is started.

   is called when the batch is started. notice the intersection is not necessarily the same as [startBatch..endBatch), since there could be only part of the data available given a requested time range. It returns a batch token of typ T, which will be provided when the job is completed or failed for checkpointing.

   Definition Classes

   HDFSCheckpointStore → CheckpointStore

7. def checkpointFailure(runningBatches: Iterable[BatchID], err: Throwable): Unit

   

   is called when the scalding job failed

   is called when the scalding job failed

   Definition Classes

   HDFSCheckpointStore → CheckpointStore

8. def checkpointPlanFailure(err: Throwable): Nothing

   

   is called when the planning is failed to start the scalding job

   is called when the planning is failed to start the scalding job

   Definition Classes

   HDFSCheckpointStore → CheckpointStore

9. def checkpointSuccessfulRun(runningBatches: Iterable[BatchID]): Unit

   

   is called when the batches are finished successfully

   is called when the batches are finished successfully

   Definition Classes

   HDFSCheckpointStore → CheckpointStore

10. def clone(): AnyRef

    

    Attributes

    protected[java.lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

11. val config: Config

    

12. val endBatch: ExclusiveUpper[BatchID]

    

    usually it's startBatch + numBatchesToRun, it's used to calculate requested time interval for the job

    usually it's startBatch + numBatchesToRun, it's used to calculate requested time interval for the job

    Definition Classes

    HDFSCheckpointStore → CheckpointStore

13. final def eq(arg0: AnyRef): Boolean

    

    Definition Classes

    AnyRef

14. def equals(arg0: Any): Boolean

    

    Definition Classes

    AnyRef → Any

15. def finalize(): Unit

    

    Attributes

    protected[java.lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( classOf[java.lang.Throwable] )

16. final def getClass(): Class[_]

    

    Definition Classes

    AnyRef → Any

17. def hashCode(): Int

    

    Definition Classes

    AnyRef → Any

18. final def isInstanceOf[T0]: Boolean

    

    Definition Classes

    Any

19. final def ne(arg0: AnyRef): Boolean

    

    Definition Classes

    AnyRef

20. final def notify(): Unit

    

    Definition Classes

    AnyRef

21. final def notifyAll(): Unit

    

    Definition Classes

    AnyRef

22. val startBatch: InclusiveLower[BatchID]

    

    should get startBatch by checking the previous batch run, it's used to calculate requested time interval for the job

    should get startBatch by checking the previous batch run, it's used to calculate requested time interval for the job

    Definition Classes

    HDFSCheckpointStore → CheckpointStore

23. final def synchronized[T0](arg0: ⇒ T0): T0

    

    Definition Classes

    AnyRef

24. def toString(): String

    

    Definition Classes

    AnyRef → Any

25. lazy val versionedStore: FileVersionTracking

    

    Attributes

    protected

26. final def wait(): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

27. final def wait(arg0: Long, arg1: Int): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

28. final def wait(arg0: Long): Unit

    

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )