Package com.yahoo.documentapi

Class ProgressToken

java.lang.Object
com.yahoo.documentapi.ProgressToken
public final class ProgressToken extends Object
Token to use to keep track of progress for visiting. Can be used to resume visiting if visiting has been aborted for any reason.
Author:
Thomas Gundersen, vekterli

  • Field Details

    • NULL_BUCKET

      public static final com.yahoo.document.BucketId NULL_BUCKET

    • FINISHED_BUCKET

      public static final com.yahoo.document.BucketId FINISHED_BUCKET

  • Constructor Details

    • ProgressToken

      public ProgressToken()
      Creates a progress token.

    • ProgressToken

      public ProgressToken(int distributionBits)

    • ProgressToken

      public ProgressToken(String serialized)

    • ProgressToken

      public ProgressToken(byte[] serialized)

  • Method Details

    • serialize

      public byte[] serialize()

    • serializeToString

      public String serializeToString()
      Returns a string (base64) encoding of the serial form of this token

    • fromSerializedString

      public static ProgressToken fromSerializedString(String serializedString)

    • addFailedBucket

      public void addFailedBucket(com.yahoo.document.BucketId superbucket, com.yahoo.document.BucketId progress, String errorMsg)

    • getFailedBuckets

      public Map<com.yahoo.document.BucketId,com.yahoo.document.BucketId> getFailedBuckets()
      Get all failed buckets and their progress. Not thread safe.
      Returns:
      Unmodifiable map of all failed buckets

    • makeNthBucketKey

      public static long makeNthBucketKey(long n, int distributionBits)
      Directly generate a bucket Id key for the nth bucket in reverse sorted order.
      Parameters:
      n - a number in the range [0, 2**distributionBits)
      distributionBits - Distribution bit count for the generated key
      Returns:
      A value where, if you had generated 2**distributionBits BucketIds with incremental numerical IDs and then sorted them on their reverse bit-order keys, the returned key would be equal to the nth element in the resulting sorted sequence.

    • getDistributionBitCount

      public int getDistributionBitCount()

    • getActiveBucketCount

      public long getActiveBucketCount()

    • getBucketCursor

      public long getBucketCursor()

    • getFinishedBucketCount

      public long getFinishedBucketCount()

    • getTotalBucketCount

      public long getTotalBucketCount()

    • getPendingBucketCount

      public long getPendingBucketCount()

    • hasPending

      public boolean hasPending()

    • hasActive

      public boolean hasActive()

    • isFinished

      public boolean isFinished()

    • isEmpty

      public boolean isEmpty()

    • getFirstErrorMsg

      public String getFirstErrorMsg()

    • containsFailedBuckets

      public boolean containsFailedBuckets()

    • isInconsistentState

      public boolean isInconsistentState()

    • setInconsistentState

      public void setInconsistentState(boolean inconsistentState)

    • toString

      public String toString()
      The format of the bucket progress output is as follows: 
         VDS bucket progress file (n% completed)\n
   distribution bit count\n
   current bucket cursor\n
   number of finished buckets\n
   total number of buckets\n
   hex-of-superbucket:hex-of-progress\n
   ... repeat above line for each pending bucket ...
      Note that unlike earlier versions of ProgressToken, the bucket IDs are not prefixed with '0x'.
      Overrides:
      toString in class Object

    • percentFinished

      public double percentFinished()
      Calculate an estimate on how far we've managed to iterate over both the superbuckets and the sub-buckets. Runs in O(n+m) time, where n is the number of active buckets and m is the number of pending buckets. Both these values should be fairly small in practice, however. Method is synchronized, as legacy code treats this as an atomic read.
      Returns:
      A value in the range [0, 100] estimating the progress.

    • bucketToKey

      public static long bucketToKey(long id)

    • keyToBucketId

      public static long keyToBucketId(long key)

    • progressFraction

      public double progressFraction(com.yahoo.document.BucketId superbucket, com.yahoo.document.BucketId progress)
      Parameters:
      superbucket - The superbucket of which progress is a sub-bucket
      progress - The sub-bucket for which a fractional progress should be calculated
      Returns:
      a value in [0, 1] specifying how far the (sub-bucket) has reached in its superbucket. This is calculated by looking at the bucket's split factor.