com.google.api.services.dataflow.model

Class Source

java.lang.Object

java.util.AbstractMap<String,Object>

com.google.api.client.util.GenericData

com.google.api.client.json.GenericJson

com.google.api.services.dataflow.model.Source

All Implemented Interfaces:

Cloneable, Map<String,Object>

public final class Source
extends GenericJson

A source that records can be read and decoded from.

This is the Java data model class that specifies how to parse/serialize into the JSON that is transmitted over HTTP when working with the Dataflow API. For a detailed explanation see: https://developers.google.com/api-client-library/java/google-http-java-client/json

Author:

Google, Inc.

Nested Class Summary

Nested classes/interfaces inherited from class com.google.api.client.util.GenericData

GenericData.Flags

Nested classes/interfaces inherited from class java.util.AbstractMap

AbstractMap.SimpleEntry<K,V>, AbstractMap.SimpleImmutableEntry<K,V>

Nested classes/interfaces inherited from interface java.util.Map

Map.Entry<K,V>

Constructor Summary

Constructors

Constructor and Description
Source()

Method Summary

Modifier and Type	Method and Description
Source	clone()
List<Map<String,Object>>	getBaseSpecs() While splitting, sources may specify the produced bundles as differences against another source, in order to save backend-side memory and allow bigger jobs.
Map<String,Object>	getCodec() The codec to use to decode data read from the source.
Boolean	getDoesNotNeedSplitting() Setting this value to true hints to the framework that the source doesn't need splitting, and using SourceSplitRequest on it would yield SOURCE_SPLIT_OUTCOME_USE_CURRENT.
SourceMetadata	getMetadata() Optionally, metadata for this source can be supplied right away, avoiding a SourceGetMetadataOperation roundtrip (see SourceOperationRequest).
Map<String,Object>	getSpec() The source to read from, plus its parameters.
Source	set(String fieldName, Object value)
Source	setBaseSpecs(List<Map<String,Object>> baseSpecs) While splitting, sources may specify the produced bundles as differences against another source, in order to save backend-side memory and allow bigger jobs.
Source	setCodec(Map<String,Object> codec) The codec to use to decode data read from the source.
Source	setDoesNotNeedSplitting(Boolean doesNotNeedSplitting) Setting this value to true hints to the framework that the source doesn't need splitting, and using SourceSplitRequest on it would yield SOURCE_SPLIT_OUTCOME_USE_CURRENT.
Source	setMetadata(SourceMetadata metadata) Optionally, metadata for this source can be supplied right away, avoiding a SourceGetMetadataOperation roundtrip (see SourceOperationRequest).
Source	setSpec(Map<String,Object> spec) The source to read from, plus its parameters.

Methods inherited from class com.google.api.client.json.GenericJson

getFactory, setFactory, toPrettyString, toString

Methods inherited from class com.google.api.client.util.GenericData

entrySet, equals, get, getClassInfo, getUnknownKeys, hashCode, put, putAll, remove, setUnknownKeys

Methods inherited from class java.util.AbstractMap

clear, containsKey, containsValue, isEmpty, keySet, size, values

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface java.util.Map

compute, computeIfAbsent, computeIfPresent, forEach, getOrDefault, merge, putIfAbsent, remove, replace, replace, replaceAll

Constructor Detail

Source

public Source()

Method Detail

getBaseSpecs

public List<Map<String,Object>> getBaseSpecs()

While splitting, sources may specify the produced bundles as differences against another source, in order to save backend-side memory and allow bigger jobs. For details, see SourceSplitRequest. To support this use case, the full set of parameters of the source is logically obtained by taking the latest explicitly specified value of each parameter in the order: base_specs (later items win), spec (overrides anything in base_specs).

Returns:

value or null for none

setBaseSpecs

public Source setBaseSpecs(List<Map<String,Object>> baseSpecs)

While splitting, sources may specify the produced bundles as differences against another source, in order to save backend-side memory and allow bigger jobs. For details, see SourceSplitRequest. To support this use case, the full set of parameters of the source is logically obtained by taking the latest explicitly specified value of each parameter in the order: base_specs (later items win), spec (overrides anything in base_specs).

Parameters:

baseSpecs - baseSpecs or null for none

getCodec

public Map<String,Object> getCodec()

The codec to use to decode data read from the source.

Returns:

value or null for none

setCodec

public Source setCodec(Map<String,Object> codec)

The codec to use to decode data read from the source.

Parameters:

codec - codec or null for none

getDoesNotNeedSplitting

public Boolean getDoesNotNeedSplitting()

Setting this value to true hints to the framework that the source doesn't need splitting, and using SourceSplitRequest on it would yield SOURCE_SPLIT_OUTCOME_USE_CURRENT. E.g. a file splitter may set this to true when splitting a single file into a set of byte ranges of appropriate size, and set this to false when splitting a filepattern into individual files. However, for efficiency, a file splitter may decide to produce file subranges directly from the filepattern to avoid a splitting round-trip. See SourceSplitRequest for an overview of the splitting process. This field is meaningful only in the Source objects populated by the user (e.g. when filling in a DerivedSource). Source objects supplied by the framework to the user don't have this field populated.

Returns:

value or null for none

setDoesNotNeedSplitting

public Source setDoesNotNeedSplitting(Boolean doesNotNeedSplitting)

Setting this value to true hints to the framework that the source doesn't need splitting, and using SourceSplitRequest on it would yield SOURCE_SPLIT_OUTCOME_USE_CURRENT. E.g. a file splitter may set this to true when splitting a single file into a set of byte ranges of appropriate size, and set this to false when splitting a filepattern into individual files. However, for efficiency, a file splitter may decide to produce file subranges directly from the filepattern to avoid a splitting round-trip. See SourceSplitRequest for an overview of the splitting process. This field is meaningful only in the Source objects populated by the user (e.g. when filling in a DerivedSource). Source objects supplied by the framework to the user don't have this field populated.

Parameters:

doesNotNeedSplitting - doesNotNeedSplitting or null for none

getMetadata

public SourceMetadata getMetadata()

Optionally, metadata for this source can be supplied right away, avoiding a SourceGetMetadataOperation roundtrip (see SourceOperationRequest). This field is meaningful only in the Source objects populated by the user (e.g. when filling in a DerivedSource). Source objects supplied by the framework to the user don't have this field populated.

Returns:

value or null for none

setMetadata

public Source setMetadata(SourceMetadata metadata)

Optionally, metadata for this source can be supplied right away, avoiding a SourceGetMetadataOperation roundtrip (see SourceOperationRequest). This field is meaningful only in the Source objects populated by the user (e.g. when filling in a DerivedSource). Source objects supplied by the framework to the user don't have this field populated.

Parameters:

metadata - metadata or null for none

getSpec

public Map<String,Object> getSpec()

The source to read from, plus its parameters.

Returns:

value or null for none

setSpec

public Source setSpec(Map<String,Object> spec)

The source to read from, plus its parameters.

Parameters:

spec - spec or null for none

set

public Source set(String fieldName,
                  Object value)

Overrides:

set in class GenericJson

clone

public Source clone()

Overrides:

clone in class GenericJson