Coders
to specify how data is encoded to and decoded from byte strings.See: Description
| Interface | Description |
|---|---|
| Coder<T> |
A
Coder<T> defines how to encode and decode values of type T into byte streams. |
| CoderFactory |
A
CoderFactory creates coders and decomposes values. |
| CoderProvider |
A
CoderProvider may create a Coder for
any concrete class. |
| DelegateCoder.CodingFunction<InputT,OutputT> |
A
CodingFunction<InputT, OutputT> is a serializable function
from InputT to OutputT that
may throw any Exception. |
| Class | Description |
|---|---|
| AtomicCoder<T> | |
| AvroCoder<T> |
A
Coder using Avro binary format. |
| AvroCoder.AvroDeterminismChecker |
Helper class encapsulating the various pieces of state maintained by the
recursive walk used for checking if the encoding will be deterministic.
|
| BigEndianIntegerCoder |
A
BigEndianIntegerCoder encodes Integers in 4 bytes, big-endian. |
| BigEndianLongCoder |
A
BigEndianLongCoder encodes Longs in 8 bytes, big-endian. |
| ByteArrayCoder |
A
ByteArrayCoder encodes byte[] objects. |
| Coder.Context |
The context in which encoding or decoding is being done.
|
| Coder.NonDeterministicException |
Exception thrown by
Coder.verifyDeterministic() if the encoding is
not deterministic. |
| CoderFactories |
Static utility methods for creating and working with
Coders. |
| CoderProviders |
A
CoderProvider may create a Coder for
any concrete class. |
| CoderRegistry | |
| CollectionCoder<T> |
A
CollectionCoder encodes Collections. |
| CustomCoder<T> |
An abstract base class for writing
Coders that encodes itself via java
serialization. |
| DelegateCoder<T,IntermediateT> |
A
DelegateCoder<T, IntermediateT> wraps a Coder for IntermediateT and
encodes/decodes values of type Ts by converting
to/from IntermediateT and then encoding/decoding using the underlying
Coder<IntermediateT>. |
| DeterministicStandardCoder<T> |
A
DeterministicStandardCoder is a StandardCoder that is
deterministic, in the sense that for objects considered equal
according to Object.equals(), the encoded bytes are
also equal. |
| DoubleCoder |
A
DoubleCoder encodes Double values in 8 bytes using Java serialization. |
| DurationCoder |
A
Coder for a joda Duration. |
| EntityCoder | |
| InstantCoder | |
| IterableCoder<T> |
An
IterableCoder encodes any Iterable
as the sequence of its elements, encoded according to the
component coder. |
| IterableLikeCoder<T,IterableT extends Iterable<T>> |
An abstract base class with functionality for assembling a
Coder for a class that implements Iterable. |
| JAXBCoder<T> |
A coder for JAXB annotated objects.
|
| KvCoder<K,V> |
A
KvCoder encodes KVs. |
| KvCoderBase<T> |
A abstract base class for KvCoder.
|
| ListCoder<T> |
A
ListCoder encodes Lists. |
| MapCoder<K,V> |
A
MapCoder encodes Maps. |
| MapCoderBase<T> |
A abstract base class for MapCoder.
|
| NullableCoder<T> |
A
NullableCoder encodes nullable values of type T using a nested
Coder<T> that does not tolerate null values. |
| Proto2Coder<T extends com.google.protobuf.Message> |
An encoder using Google Protocol Buffers 2 binary format.
|
| SerializableCoder<T extends Serializable> | |
| SetCoder<T> |
A
SetCoder<T> encodes any Set<T>
as an encoding of an iterable of its elements. |
| StandardCoder<T> |
A
StandardCoder is a Coder that defines equality,
hashing, and printing via the class name and recursively using
StandardCoder.getComponents(). |
| StringDelegateCoder<T> |
A
StringDelegateCoder<T> wraps a Coder<String>
and encodes/decodes values of type T via string representations. |
| StringUtf8Coder |
A
StringUtf8Coder encodes Java Strings in UTF-8 encoding. |
| TableRowJsonCoder |
A
TableRowJsonCoder encodes BigQuery TableRow objects. |
| TextualIntegerCoder |
A
TextualIntegerCoder encodes Integers
as their textual, decimal, representation. |
| VarIntCoder |
A
VarIntCoder encodes Integers using between 1 and 5 bytes. |
| VarLongCoder |
A
VarLongCoder encodes longs using between 1 and 10 bytes. |
| VoidCoder |
A
VoidCoder encodes Voids. |
| Enum | Description |
|---|---|
| CannotProvideCoderException.ReasonCode |
Indicates the reason that
Coder inference failed. |
| Exception | Description |
|---|---|
| CannotProvideCoderException |
The exception thrown when a
CoderProvider cannot
provide a Coder that has been requested. |
| CoderException |
A
CoderException is thrown if there is a problem encoding or
decoding a value. |
| Annotation Type | Description |
|---|---|
| DefaultCoder |
The
DefaultCoder annotation
specifies a default Coder class to handle encoding and decoding
instances of the annotated class. |
Coders
to specify how data is encoded to and decoded from byte strings.
During execution of a Pipeline, elements in a
PCollection
may need to be encoded into byte strings.
This happens both at the beginning and end of a pipeline when data is read from and written to
persistent storage and also during execution of a pipeline when elements are communicated between
machines.
Exactly when PCollection elements are encoded during execution depends on which
PipelineRunner is being used and how that runner
chooses to execute the pipeline. As such, Dataflow requires that all PCollections have an
appropriate Coder in case it becomes necessary. In many cases, the Coder can be inferred from
the available Java type
information and the Pipeline's CoderRegistry. It
can be specified per PCollection via
PCollection.setCoder(Coder) or per type using the
DefaultCoder annotation.
This package provides a number of coders for common types like Integer,
String, and List, as well as coders like
AvroCoder that can be used to encode many custom
types.