public abstract static class DatastoreV1.Read
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,org.apache.beam.sdk.values.PCollection<com.google.datastore.v1.Entity>>
PTransform
that reads the result rows of a Cloud Datastore query as Entity
objects.DatastoreIO
,
Serialized FormModifier and Type | Field and Description |
---|---|
static int |
NUM_QUERY_SPLITS_MAX
An upper bound on the number of splits for a query.
|
Constructor and Description |
---|
Read() |
Modifier and Type | Method and Description |
---|---|
org.apache.beam.sdk.values.PCollection<com.google.datastore.v1.Entity> |
expand(org.apache.beam.sdk.values.PBegin input) |
abstract @Nullable org.apache.beam.sdk.options.ValueProvider<java.lang.String> |
getLiteralGqlQuery() |
abstract @Nullable java.lang.String |
getLocalhost() |
abstract @Nullable org.apache.beam.sdk.options.ValueProvider<java.lang.String> |
getNamespace() |
long |
getNumEntities(org.apache.beam.sdk.options.PipelineOptions options,
java.lang.String ourKind,
@Nullable java.lang.String namespace)
Returns Number of entities available for reading.
|
abstract int |
getNumQuerySplits() |
abstract @Nullable org.apache.beam.sdk.options.ValueProvider<java.lang.String> |
getProjectId() |
abstract @Nullable com.google.datastore.v1.Query |
getQuery() |
void |
populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder) |
abstract java.lang.String |
toString() |
DatastoreV1.Read |
withLiteralGqlQuery(java.lang.String gqlQuery)
Returns a new
DatastoreV1.Read that reads the results of the specified GQL query. |
DatastoreV1.Read |
withLiteralGqlQuery(org.apache.beam.sdk.options.ValueProvider<java.lang.String> gqlQuery)
Same as
withLiteralGqlQuery(String) but with a ValueProvider . |
DatastoreV1.Read |
withLocalhost(java.lang.String localhost)
Returns a new
DatastoreV1.Read that reads from a Datastore Emulator running at the
given localhost address. |
DatastoreV1.Read |
withNamespace(java.lang.String namespace)
Returns a new
DatastoreV1.Read that reads from the given namespace. |
DatastoreV1.Read |
withNamespace(org.apache.beam.sdk.options.ValueProvider<java.lang.String> namespace)
Same as
withNamespace(String) but with a ValueProvider . |
DatastoreV1.Read |
withNumQuerySplits(int numQuerySplits)
|
DatastoreV1.Read |
withProjectId(java.lang.String projectId)
Returns a new
DatastoreV1.Read that reads from the Cloud Datastore for the specified
project. |
DatastoreV1.Read |
withProjectId(org.apache.beam.sdk.options.ValueProvider<java.lang.String> projectId)
Same as
withProjectId(String) but with a ValueProvider . |
DatastoreV1.Read |
withQuery(com.google.datastore.v1.Query query)
Returns a new
DatastoreV1.Read that reads the results of the specified query. |
public static final int NUM_QUERY_SPLITS_MAX
public abstract @Nullable org.apache.beam.sdk.options.ValueProvider<java.lang.String> getProjectId()
public abstract @Nullable com.google.datastore.v1.Query getQuery()
public abstract @Nullable org.apache.beam.sdk.options.ValueProvider<java.lang.String> getLiteralGqlQuery()
public abstract @Nullable org.apache.beam.sdk.options.ValueProvider<java.lang.String> getNamespace()
public abstract int getNumQuerySplits()
public abstract @Nullable java.lang.String getLocalhost()
public abstract java.lang.String toString()
toString
in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,org.apache.beam.sdk.values.PCollection<com.google.datastore.v1.Entity>>
public DatastoreV1.Read withProjectId(java.lang.String projectId)
DatastoreV1.Read
that reads from the Cloud Datastore for the specified
project.public DatastoreV1.Read withProjectId(org.apache.beam.sdk.options.ValueProvider<java.lang.String> projectId)
withProjectId(String)
but with a ValueProvider
.public DatastoreV1.Read withQuery(com.google.datastore.v1.Query query)
DatastoreV1.Read
that reads the results of the specified query.
Note: Normally, DatastoreIO
will read from Cloud Datastore in parallel
across many workers. However, when the Query
is configured with a limit using Query.Builder#setLimit
, then all results will be read by a single worker in order to ensure
correct results.
public DatastoreV1.Read withLiteralGqlQuery(java.lang.String gqlQuery)
DatastoreV1.Read
that reads the results of the specified GQL query. See
GQL Reference
to know more about GQL grammar.
Note: This query is executed with literals allowed, so the users should ensure that the query is originated from trusted sources to avoid any security vulnerabilities via SQL Injection.
Cloud Datastore does not a provide a clean way to translate a gql query string to Query
, so we end up making a query to the service for translation but this may read the
actual data, although it will be a small amount. It needs more validation through production
use cases before marking it as stable.
public DatastoreV1.Read withLiteralGqlQuery(org.apache.beam.sdk.options.ValueProvider<java.lang.String> gqlQuery)
withLiteralGqlQuery(String)
but with a ValueProvider
.public DatastoreV1.Read withNamespace(java.lang.String namespace)
DatastoreV1.Read
that reads from the given namespace.public DatastoreV1.Read withNamespace(org.apache.beam.sdk.options.ValueProvider<java.lang.String> namespace)
withNamespace(String)
but with a ValueProvider
.public DatastoreV1.Read withNumQuerySplits(int numQuerySplits)
DatastoreV1.Read
that reads by splitting the given query
into
numQuerySplits
.
The semantics for the query splitting is defined below:
NUM_QUERY_SPLITS_MAX
will be capped at NUM_QUERY_SPLITS_MAX
.
query
has a user limit set, then numQuerySplits
will be ignored
and no split will be performed.
public DatastoreV1.Read withLocalhost(java.lang.String localhost)
DatastoreV1.Read
that reads from a Datastore Emulator running at the
given localhost address.public long getNumEntities(org.apache.beam.sdk.options.PipelineOptions options, java.lang.String ourKind, @Nullable java.lang.String namespace)
public org.apache.beam.sdk.values.PCollection<com.google.datastore.v1.Entity> expand(org.apache.beam.sdk.values.PBegin input)
expand
in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,org.apache.beam.sdk.values.PCollection<com.google.datastore.v1.Entity>>
public void populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)
populateDisplayData
in interface org.apache.beam.sdk.transforms.display.HasDisplayData
populateDisplayData
in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,org.apache.beam.sdk.values.PCollection<com.google.datastore.v1.Entity>>