com.google.cloud.dataflow.sdk.io

Class DatastoreIO.Source

java.lang.Object

com.google.cloud.dataflow.sdk.io.Source<T>

com.google.cloud.dataflow.sdk.io.BoundedSource<DatastoreV1.Entity>

com.google.cloud.dataflow.sdk.io.DatastoreIO.Source

All Implemented Interfaces:

Serializable

Enclosing class:

DatastoreIO

public static class DatastoreIO.Source
extends BoundedSource<DatastoreV1.Entity>

A Source that reads the result rows of a Datastore query as Entity objects.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class com.google.cloud.dataflow.sdk.io.BoundedSource

BoundedSource.BoundedReader<T>

Nested classes/interfaces inherited from class com.google.cloud.dataflow.sdk.io.Source

Source.Reader<T>

Method Summary

Modifier and Type	Method and Description
BoundedSource.BoundedReader<DatastoreV1.Entity>	createReader(PipelineOptions pipelineOptions) Returns a new BoundedSource.BoundedReader that reads from this source.
String	getDataset()
Coder<DatastoreV1.Entity>	getDefaultOutputCoder() Returns the default Coder to use for the data read from this source.
long	getEstimatedSizeBytes(PipelineOptions options) An estimate of the total size (in bytes) of the data that would be read from this source.
String	getHost()
String	getNamespace()
DatastoreV1.Query	getQuery()
boolean	producesSortedKeys(PipelineOptions options) Whether this source is known to produce key/value pairs sorted by lexicographic order on the bytes of the encoded key.
List<DatastoreIO.Source>	splitIntoBundles(long desiredBundleSizeBytes, PipelineOptions options) Splits the source into bundles of approximately desiredBundleSizeBytes.
String	toString()
void	validate() Checks that this source is valid, before it can be used in a pipeline.
DatastoreIO.Source	withDataset(String datasetId)
DatastoreIO.Source	withHost(String host)
DatastoreIO.Source	withNamespace(String namespace)
DatastoreIO.Source	withQuery(DatastoreV1.Query query)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

getHost

public String getHost()

getDataset

public String getDataset()

getQuery

public DatastoreV1.Query getQuery()

getNamespace

@Nullable
public String getNamespace()

withDataset

public DatastoreIO.Source withDataset(String datasetId)

withQuery

public DatastoreIO.Source withQuery(DatastoreV1.Query query)

withHost

public DatastoreIO.Source withHost(String host)

withNamespace

public DatastoreIO.Source withNamespace(@Nullable
                                        String namespace)

getDefaultOutputCoder

public Coder<DatastoreV1.Entity> getDefaultOutputCoder()

Description copied from class: Source

Returns the default Coder to use for the data read from this source.

Specified by:

getDefaultOutputCoder in class Source<DatastoreV1.Entity>

producesSortedKeys

public boolean producesSortedKeys(PipelineOptions options)

Description copied from class: BoundedSource

Whether this source is known to produce key/value pairs sorted by lexicographic order on the bytes of the encoded key.

Specified by:

producesSortedKeys in class BoundedSource<DatastoreV1.Entity>

splitIntoBundles

public List<DatastoreIO.Source> splitIntoBundles(long desiredBundleSizeBytes,
                                                 PipelineOptions options)
                                          throws Exception

Description copied from class: BoundedSource

Splits the source into bundles of approximately desiredBundleSizeBytes.

Specified by:

splitIntoBundles in class BoundedSource<DatastoreV1.Entity>

Throws:

Exception

createReader

public BoundedSource.BoundedReader<DatastoreV1.Entity> createReader(PipelineOptions pipelineOptions)
                                                             throws IOException

Description copied from class: BoundedSource

Returns a new BoundedSource.BoundedReader that reads from this source.

Specified by:

createReader in class BoundedSource<DatastoreV1.Entity>

Throws:

IOException

validate

public void validate()

Description copied from class: Source

Checks that this source is valid, before it can be used in a pipeline.

It is recommended to use Preconditions for implementing this method.

Specified by:

validate in class Source<DatastoreV1.Entity>

getEstimatedSizeBytes

public long getEstimatedSizeBytes(PipelineOptions options)
                           throws Exception

Description copied from class: BoundedSource

An estimate of the total size (in bytes) of the data that would be read from this source. This estimate is in terms of external storage size, before any decompression or other processing done by the reader.

Specified by:

getEstimatedSizeBytes in class BoundedSource<DatastoreV1.Entity>

Throws:

Exception

toString

public String toString()

Overrides:

toString in class Object