DatastoreIO.DatastoreReader (Google Cloud Dataflow SDK 1.3.0 API)

java.lang.Object
- com.google.cloud.dataflow.sdk.io.Source.Reader<T>
- - com.google.cloud.dataflow.sdk.io.BoundedSource.BoundedReader<DatastoreV1.Entity>
  - - com.google.cloud.dataflow.sdk.io.DatastoreIO.DatastoreReader

All Implemented Interfaces:

AutoCloseable

Enclosing class:

DatastoreIO
```
public static class DatastoreIO.DatastoreReader
extends BoundedSource.BoundedReader<DatastoreV1.Entity>
```
A Source.Reader over the records from a query of the datastore.
Timestamped records are currently not supported. All records implicitly have the timestamp of BoundedWindow.TIMESTAMP_MIN_VALUE.

Constructor Summary

Constructors
Constructor and Description

DatastoreReader(DatastoreIO.Source source, Datastore datastore)
Returns a DatastoreReader with Source and Datastore object set.

Constructors
Constructor and Description
`DatastoreReader(DatastoreIO.Source source, Datastore datastore)` Returns a DatastoreReader with Source and Datastore object set.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`boolean`	`advance()` Advances the reader to the next valid record.
`void`	`close()` Closes the reader.
`DatastoreV1.Entity`	`getCurrent()` Returns the value of the data item that was read by the last `Source.Reader.start()` or `Source.Reader.advance()` call.
`DatastoreIO.Source`	`getCurrentSource()` Returns a `Source` describing the same input that this `Reader` reads (including items already read).
`Double`	`getFractionConsumed()` Returns a value in [0, 1] representing approximately what fraction of the `current source` this reader has read so far, or `null` if such an estimate is not available.
`DatastoreIO.Source`	`splitAtFraction(double fraction)` Tells the reader to narrow the range of the input it's going to read and give up the remainder, so that the new range would contain approximately the given fraction of the amount of data in the current range.
`boolean`	`start()` Initializes the reader and advances the reader to the first record.

Methods inherited from class com.google.cloud.dataflow.sdk.io.BoundedSource.BoundedReader
getCurrentTimestamp

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - DatastoreReader
```
public DatastoreReader(DatastoreIO.Source source,
                       Datastore datastore)
```
    Returns a DatastoreReader with Source and Datastore object set.
    
    Parameters:
    
    datastore - a datastore connection to use.
- Method Detail
  - getCurrent
```
public DatastoreV1.Entity getCurrent()
```
    Description copied from class: Source.Reader
    
    Returns the value of the data item that was read by the last Source.Reader.start() or Source.Reader.advance() call. The returned value must be effectively immutable and remain valid indefinitely.
    Multiple calls to this method without an intervening call to Source.Reader.advance() should return the same result.
    
    Specified by:
    
    getCurrent in class Source.Reader<DatastoreV1.Entity>
  - start
```
public boolean start()
              throws IOException
```
    Description copied from class: Source.Reader
    
    Initializes the reader and advances the reader to the first record.
    This method should be called exactly once. The invocation should occur prior to calling Source.Reader.advance() or Source.Reader.getCurrent(). This method may perform expensive operations that are needed to initialize the reader.
    
    Specified by:
    
    start in class Source.Reader<DatastoreV1.Entity>
    
    Returns:
    
    true if a record was read, false if there is no more input available.
    
    Throws:
    
    IOException
  - advance
```
public boolean advance()
                throws IOException
```
    Description copied from class: Source.Reader
    
    Advances the reader to the next valid record.
    
    Specified by:
    
    advance in class Source.Reader<DatastoreV1.Entity>
    
    Returns:
    
    true if a record was read, false if there is no more input available.
    
    Throws:
    
    IOException
  - close
```
public void close()
           throws IOException
```
    Description copied from class: Source.Reader
    
    Closes the reader. The reader cannot be used after this method is called.
    
    Specified by:
    
    close in interface AutoCloseable
    
    Specified by:
    
    close in class Source.Reader<DatastoreV1.Entity>
    
    Throws:
    
    IOException
  - getCurrentSource
```
public DatastoreIO.Source getCurrentSource()
```
    Description copied from class: Source.Reader
    
    Returns a Source describing the same input that this Reader reads (including items already read).
    A reader created from the result of getCurrentSource, if consumed, MUST return the same data items as the current reader.
    
    Specified by:
    
    getCurrentSource in class BoundedSource.BoundedReader<DatastoreV1.Entity>
  - splitAtFraction
```
public DatastoreIO.Source splitAtFraction(double fraction)
```
    Description copied from class: BoundedSource.BoundedReader
    Tells the reader to narrow the range of the input it's going to read and give up the remainder, so that the new range would contain approximately the given fraction of the amount of data in the current range.
    Returns a BoundedSource representing the remainder.
    Detailed description
    Assuming the following sequence of calls:
```
   BoundedSource<T> initial = reader.getCurrentSource();
   BoundedSource<T> residual = reader.splitAtFraction(fraction);
   BoundedSource<T> primary = reader.getCurrentSource();
 
```
    - The "primary" and "residual" sources, when read, should together cover the same set of records as "initial".
    - The current reader should continue to be in a valid state, and continuing to read from it should, together with the records it already read, yield the same records as would have been read by "primary".
    - The amount of data read by "primary" should ideally represent approximately the given fraction of the amount of data read by "initial".
    For example, a reader that reads a range of offsets [A, B) in a file might implement this method by truncating the current range to [A, A + fraction*(B-A)) and returning a Source representing the range [A + fraction*(B-A), B).
    This method should return null if the split cannot be performed for this fraction while satisfying the semantics above. E.g., a reader that reads a range of offsets in a file should return null if it is already past the position in its range corresponding to the given fraction. In this case, the method MUST have no effect (the reader must behave as if the method hadn't been called at all).
    Statefulness
    Since this method (if successful) affects the reader's source, in subsequent invocations "fraction" should be interpreted relative to the new current source.
    Thread safety and blocking
    This method will be called concurrently to other methods (however there will not be multiple concurrent invocations of this method itself), and it is critical for it to be implemented in a thread-safe way (otherwise data loss is possible).
    It is also very important that this method always completes quickly. In particular, it should not perform or wait on any blocking operations such as I/O, RPCs etc. Violating this requirement may stall completion of the work item or even cause it to fail.
    It is incorrect to make both this method and Source.Reader.start()/Source.Reader.advance() synchronized, because those methods can perform blocking operations, and then this method would have to wait for those calls to complete.
    RangeTracker makes it easy to implement this method safely and correctly.
    By default, returns null to indicate that splitting is not possible.
    Overrides:
    
    splitAtFraction in class BoundedSource.BoundedReader<DatastoreV1.Entity>
  - getFractionConsumed
```
public Double getFractionConsumed()
```
    Description copied from class: BoundedSource.BoundedReader
    Returns a value in [0, 1] representing approximately what fraction of the current source this reader has read so far, or null if such an estimate is not available.
    It is recommended that this method should satisfy the following properties:
    - Should return 0 before the Source.Reader.start() call.
    - Should return 1 after a Source.Reader.start() or Source.Reader.advance() call that returns false.
    - The returned values should be non-decreasing (though they don't have to be unique).
    By default, returns null to indicate that this cannot be estimated.
    Thread safety
    If BoundedSource.BoundedReader.splitAtFraction(double) is implemented, this method can be called concurrently to other methods (including itself), and it is therefore critical for it to be implemented in a thread-safe way.
    Overrides:
    
    getFractionConsumed in class BoundedSource.BoundedReader<DatastoreV1.Entity>

Class DatastoreIO.DatastoreReader

Constructor Summary

Method Summary

Methods inherited from class com.google.cloud.dataflow.sdk.io.BoundedSource.BoundedReader

Methods inherited from class java.lang.Object

Constructor Detail

DatastoreReader

Method Detail

getCurrent

start

advance

close

getCurrentSource

splitAtFraction

Detailed description

Statefulness

Thread safety and blocking

getFractionConsumed

Thread safety