Coders
to specify how data is encoded to and decoded from byte strings.See: Description
| Interface | Description |
|---|---|
| Coder<T> |
A
Coder<T> defines how to encode and decode values of type T into byte streams. |
| CoderFactory |
A
CoderFactory creates coders and decomposes values. |
| CoderProvider |
A
CoderProvider may create a Coder for
any concrete class. |
| DelegateCoder.CodingFunction<Input,Output> |
A
CodingFunction<Input, Output> is a serializable function
from Input to Output that
may throw any Exception. |
| Class | Description |
|---|---|
| AtomicCoder<T> |
An AtomicCoder is one that has no component Coders or other state.
|
| AvroCoder<T> |
An encoder using Avro binary format.
|
| AvroCoder.AvroDeterminismChecker |
Helper class encapsulating the various pieces of state maintained by the
recursive walk used for checking if the encoding will be deterministic.
|
| BigEndianIntegerCoder |
A BigEndianIntegerCoder encodes Integers in 4 bytes, big-endian.
|
| BigEndianLongCoder |
A BigEndianLongCoder encodes Longs in 8 bytes, big-endian.
|
| ByteArrayCoder |
A ByteArrayCoder encodes byte[] objects.
|
| Coder.Context |
The context in which encoding or decoding is being done.
|
| Coder.NonDeterministicException |
Exception thrown by
Coder.verifyDeterministic() if the encoding is
not deterministic. |
| CoderRegistry |
A CoderRegistry allows registering the default Coder to use for a Java class,
and looking up and instantiating the default Coder for a Java type.
|
| CollectionCoder<T> |
A CollectionCoder encodes Collections.
|
| CustomCoder<T> |
An abstract base class for writing Coders that encodes itself via java
serialization.
|
| DelegateCoder<T,DT> | |
| DoubleCoder |
A DoubleCoder encodes Doubles in 8 bytes.
|
| EntityCoder |
An EntityCoder encodes/decodes Datastore Entity objects.
|
| InstantCoder |
A InstantCoder encodes joda Instant.
|
| IterableCoder<T> |
An IterableCoder encodes Iterables.
|
| IterableLikeCoder<T,IT extends java.lang.Iterable<T>> |
The base class of Coders for Iterable subclasses.
|
| JAXBCoder<T> |
A coder for JAXB annotated objects.
|
| KvCoder<K,V> |
A KvCoder encodes KVs.
|
| KvCoderBase<T> |
A abstract base class for KvCoder.
|
| ListCoder<T> |
A ListCoder encodes Lists.
|
| MapCoder<K,V> |
A MapCoder encodes Maps.
|
| MapCoderBase<T> |
A abstract base class for MapCoder.
|
| Proto2Coder<T extends com.google.protobuf.Message> |
An encoder using Google Protocol Buffers 2 binary format.
|
| SerializableCoder<T extends java.io.Serializable> |
An encoder of
Serializable objects. |
| SetCoder<T> |
A SetCoder encodes Sets.
|
| StandardCoder<T> |
A StandardCoder is one that defines equality, hashing, and printing
via the class name and recursively using
StandardCoder.getComponents(). |
| StringDelegateCoder<T> |
A
StringDelegateCoder<T> wraps a Coder<String>
and encodes/decodes values of type T via string representations. |
| StringUtf8Coder |
A StringUtf8Coder encodes Java Strings in UTF-8 encoding.
|
| TableRowJsonCoder |
A TableRowJsonCoder encodes BigQuery TableRow objects.
|
| TextualIntegerCoder |
A TextualIntegerCoder encodes Integers as text.
|
| VarIntCoder |
A VarIntCoder encodes Integers using between 1 and 5 bytes.
|
| VarLongCoder |
A VarLongCoder encodes longs using between 1 and 10 bytes.
|
| VoidCoder |
A VoidCoder encodes Voids.
|
| Exception | Description |
|---|---|
| CoderException |
A CoderException is thrown if there is a problem encoding or
decoding a value.
|
| Annotation Type | Description |
|---|---|
| DefaultCoder |
Specifies a default
Coder class to handle encoding and decoding
instances of the annotated class. |
Coders
to specify how data is encoded to and decoded from byte strings.
During execution of a Pipeline, elements in a
PCollection
may need to be encoded into byte strings.
This happens both at the beginning and end of a pipeline when data is read from and written to
persistent storage and also during execution of a pipeline when elements are communicated between
machines.
Exactly when PCollection elements are encoded during execution depends on which
PipelineRunner is being used and how that runner
chooses to execute the pipeline. As such, Dataflow requires that all PCollections have an
appropriate Coder in case it becomes necessary. In many cases, the Coder can be inferred from
the available Java type
information and the Pipeline's CoderRegistry. It
can be specified per PCollection via
PCollection.setCoder(Coder) or per type using the
DefaultCoder annotation.
This package provides a number of coders for common types like Integer,
String, and List, as well as coders like
AvroCoder that can be used to encode many custom
types.