Type Members

1.  case class Anno(className: String, args: Map[String, String]) extends Product with Serializable
2.  class AnnotationExtractors extends AnyRef
3.  class Avro4sConfigurationException extends Avro4sException
4.  class Avro4sDecodingException extends Avro4sException
5.  class Avro4sEncodingException extends Avro4sException
6.  class Avro4sException extends Exception

7.  case class AvroAlias(alias: String) extends Annotation with AvroAliasable with Product with Serializable
8.  trait AvroAliasable extends Annotation with AvroFieldReflection
9.  class AvroBinaryInputStream[T] extends AvroInputStream[T]

   An implementation of AvroInputStream that reads values of type T written as binary data.

   An implementation of AvroInputStream that reads values of type T written as binary data. See https://avro.apache.org/docs/current/spec.html#binary_encoding

   In order to convert the underlying binary data into types of T, this input stream requires an instance of Decoder.

10.  class AvroDataInputStream[T] extends AvroInputStream[T]
11.  case class AvroDataOutputStream[T](os: OutputStream, codec: CodecFactory)(implicit encoder: Encoder[T]) extends AvroOutputStream[T] with Product with Serializable

    An AvroOutputStream that writes the schema along with the messages.

    An AvroOutputStream that writes the schema along with the messages. This is usually the format required when writing multiple messages to a single file. Some frameworks, such as a Kafka, store the Schema separately to messages, in which case the AvroBinaryInputStream is what you would need.

    os

    the underlying stream that data will be written to.

    codec

    compression codec

    encoder

    the avro4s Encoder that will convert each value to a GenericRecord.

12.  case class AvroDoc(doc: String) extends Annotation with AvroDocumentable with Product with Serializable
13.  trait AvroDocumentable extends Annotation with AvroFieldReflection
14.  case class AvroEnumDefault(default: Any) extends Annotation with StaticAnnotation with Product with Serializable
15.  case class AvroErasedName() extends Annotation with AvroFieldReflection with Product with Serializable

    This annotation is used to disable generics in the encoding of a record's name.

    This annotation is used to disable generics in the encoding of a record's name.

    Normally, the record name for a generic type is the name of the raw type, plus the actual type parameters. For example, a class Foo with type parameters Int and Boolean, would have a generated name of FooInt_Boolean

    When this annotation is present on a type, the name used in the schema will simply be the raw type, eg Foo.

16.  sealed trait AvroFieldReflection extends Annotation with StaticAnnotation
17.  trait AvroFixable extends Annotation with AvroFieldReflection
18.  case class AvroFixed(size: Int) extends Annotation with AvroFixable with Product with Serializable

    AvroFixed overrides the schema type for a field or a value class so that the schema is set to org.apache.avro.Schema.Type.FIXED rather than whatever the default would be.

    AvroFixed overrides the schema type for a field or a value class so that the schema is set to org.apache.avro.Schema.Type.FIXED rather than whatever the default would be.

    This annotation can be used in the following ways:

    - On a field, eg case class Foo(@AvroField(10) name: String) which results in the field name having schema type FIXED with a size of 10.

    - On a value type, eg @AvroField(7) case class Foo(name: String) extends AnyVal which results in all usages of the value type having schema FIXED with a size of 7 rather than the default.

19.  sealed trait AvroFormat extends AnyRef
20.  trait AvroInputStream[T] extends AutoCloseable
21.  class AvroInputStreamBuilder[T] extends AnyRef
22.  class AvroInputStreamBuilderWithSource[T] extends AnyRef
23.  class AvroJavaEnumDefault extends Annotation with Annotation with ClassfileAnnotation
24.  class AvroJavaName extends Annotation with Annotation with ClassfileAnnotation
25.  class AvroJavaNamespace extends Annotation with Annotation with ClassfileAnnotation
26.  class AvroJavaProp extends Annotation with Annotation with ClassfileAnnotation
27.  final case class AvroJsonInputStream[T](in: InputStream, writerSchema: Schema)(implicit decoder: Decoder[T]) extends AvroInputStream[T] with Product with Serializable
28.  case class AvroName(name: String) extends Annotation with AvroNameable with Product with Serializable

    AvroName allows the name used by Avro to be different from what is defined in code.

    AvroName allows the name used by Avro to be different from what is defined in code.

    For example, if a case class defines a field z, such as case class Foo(z: String) then normally this will be serialized as an entry 'z' in the Avro Record.

    However, if the field is annotated such as case class Foo(@AvroName("x") z: String) then the entry in the Avro Record will be for 'x'.

    Similarly for deserialization, if a field is annotated then the name that is looked up in the avro record will be the annotated name and not the field name in Scala.

    The second example is with classes. If a class is annotated with @AvroName then the name used in the record schema will not be the classname but the annotated value.

    This will also have an effect on serialization. For example, when decoding records into an Either, the decoder must decide if the value is a Left or a Right. It usually does this by comparing the name in the record to the classnames of the either types, but when annotated, it will compare the name in the record to the annotated value.

29.  trait AvroNameable extends Annotation with AvroFieldReflection
30.  case class AvroNamespace(namespace: String) extends Annotation with AvroNamespaceable with Product with Serializable
31.  trait AvroNamespaceable extends Annotation with AvroFieldReflection
32.  case class AvroNoDefault() extends Annotation with StaticAnnotation with Product with Serializable
33.  trait AvroOutputStream[T] extends AutoCloseable

    An AvroOutputStream will write instances of T to an underlying representation.

    An AvroOutputStream will write instances of T to an underlying representation.

    There are three implementations of this stream

    • a Data stream,
    • a Binary stream
    • a Json stream

    See the methods on the companion object to create instances of each of these types of stream.

34.  class AvroOutputStreamBuilder[T] extends AnyRef
35.  class AvroOutputStreamBuilderWithSource[T] extends AnyRef
36.  case class AvroProp(key: String, value: String) extends Annotation with AvroProperty with Product with Serializable
37.  trait AvroProperty extends Annotation with AvroFieldReflection
38.  case class AvroSortPriority(priority: Float) extends Annotation with AvroFieldReflection with Product with Serializable
39.  case class AvroTransient() extends Annotation with StaticAnnotation with Product with Serializable
40.  case class AvroUnionPosition(position: Int) extends Annotation with StaticAnnotation with Product with Serializable
41.  trait BaseDecoders extends AnyRef
42.  trait BaseEncoders extends AnyRef
43.  trait BaseSchemaFors extends AnyRef
44.  trait BigDecimalDecoders extends AnyRef
45.  trait BigDecimalEncoders extends AnyRef
46.  trait ByteIterableDecoders extends AnyRef
47.  trait ByteIterableEncoders extends AnyRef
48.  trait ByteIterableSchemaFors extends AnyRef
49.  sealed trait CaseClassShape extends AnyRef
50.  trait Codec[T] extends Encoder[T] with Decoder[T] with SchemaAware[Codec, T]

    Convenience trait that allows to define both custom Encoder and custom Decoder in one go, if both are needed.

    Convenience trait that allows to define both custom Encoder and custom Decoder in one go, if both are needed. This way, they don't have to be declared in two separate classes or objects.

    A codec can both encode a Scala value of type T into a compatible Avro value based on the given schema as well as decode an Avro value, such as a GenericRecord, SpecificRecord, GenericFixed, EnumSymbol, or a basic JVM type, into a target Scala type.

    For advanced users: note that there is no ResolvableCodec type. If you need to define both Encoder and Decoder for a type that is within a chain of recursive types, you need to define them in two separate classes or objects.

51.  trait CollectionAndContainerDecoders extends AnyRef
52.  trait CollectionAndContainerEncoders extends AnyRef
53.  trait CollectionAndContainerSchemaFors extends AnyRef
54.  sealed trait CustomDefault extends AnyRef
55.  case class CustomEnumDefault(value: String) extends CustomDefault with Product with Serializable
56.  case class CustomUnionDefault(className: String, values: Map[String, Any]) extends CustomDefault with Product with Serializable
57.  case class CustomUnionWithEnumDefault(parentName: String, default: String, value: String) extends CustomDefault with Product with Serializable
58.  trait Decoder[T] extends SchemaAware[Decoder, T] with Serializable

    A Decoder is used to convert an Avro value, such as a GenericRecord, SpecificRecord, GenericFixed, EnumSymbol, or a basic JVM type, into a target Scala type.

    A Decoder is used to convert an Avro value, such as a GenericRecord, SpecificRecord, GenericFixed, EnumSymbol, or a basic JVM type, into a target Scala type.

    For example, a Decoder[String] would convert an input of type Utf8 - which is one of the ways Avro can encode strings - into a plain Java String.

    Another example, a decoder for Option[String] would handle inputs of null by emitting a None, and a non-null input by emitting the decoded value wrapped in a Some.

    A final example is converting a GenericData.Array or a Java collection type into a Scala collection type.

59.  class DefaultAvroOutputStream[T] extends AvroOutputStream[T]
60.  class DefaultAwareDatumReader[T] extends GenericDatumReader[T]
61.  class DefaultAwareGenericData extends GenericData
62.  class DefinitionEnvironment[Typeclass[_]] extends AnyRef

    Helps with encoders/decoders/schemas for recursive types.

    Helps with encoders/decoders/schemas for recursive types. Provides a map of already defined bindings for types. For an Encoder e.g., captures which encoders are already defined when building the (cyclic) encoder graph for a recursive type.

    Initially, one would expect that handling recursive types can be delegated to Magnolia. Magnolia uses definitions and lazy evaluation to defer the evaluation of recursive types, so that the recursive definition of the type class gets unrolled as needed during data processing. There are two reason why this approach doesn't work in avro4s: one reason for SchemaFor, and one reason for both encoders adn decoders.

    a) For SchemaFor, the avro library requires to build a cyclic graph of records for recursive types. Lazily unfolding doesn't work here. To give an example, consider the type

    sealed trait Tree
    case class Leaf(value: Int) extends Tree
    case class Branch(left: Tree, right: Tree) extends Tree

    The Avro schema for this needs to be constructed by first creating the records, then the union, and then setting the fields of the branch record with the schema of the tree union:

    val leaf   = Schema.createRecord(...)
    val branch = Schema.createRecord(...)
    val tree   = Schema.createUnion(branch, leaf)
    branch.setFields(...) // using the tree schema just created before

    b) For encoder / decoder, leveraging Magnolias mechanism would work. However, the construction of encoders and decoders is expensive as computing name mappings involves type reflection that would be re-computed on every unfold. Additionally, annotations on one type may trickle down to other types, e.g., a namespace annotation on a trait will be applied to all case classes below it. This push-down of modifications will have to be done on every unfolding as well, meaning additional object allocations.

    So instead of using the Magnolia mechanism for encoder / decoder, we reuse our own recursive type handling machinery that we need for SchemaFor, and by this 1) stay consistent within this code base, and 2) achieve much better runtime performance on encoding and decoding (i.e. increase the throughput by roughly 10x).

    Typeclass

    Encoder, Decoder or SchemaFor

63.  trait Encoder[T] extends SchemaAware[Encoder, T] with Serializable

    An Encoder encodes a Scala value of type T into a compatible Avro value based on the given schema.

    An Encoder encodes a Scala value of type T into a compatible Avro value based on the given schema.

    For example, given a string, and a schema of type Schema.Type.STRING then the string would be encoded as an instance of Utf8, whereas the same string and a Schema.Type.FIXED would be encoded as an instance of GenericFixed.

    Another example is given a Scala enumeration value, and a schema of type Schema.Type.ENUM, the value would be encoded as an instance of GenericData.EnumSymbol.

64.  trait EnumSchemaFor extends AnyRef
65.  trait FieldMapper extends AnyRef

    We may have a schema with a field in snake case like say { "first_name": "sam" } and that schema needs to be used for a case class field firstName.

    We may have a schema with a field in snake case like say { "first_name": "sam" } and that schema needs to be used for a case class field firstName.

    The FieldMapper is used to map fields in a schema to fields in a class by transforming the class field name to the wire name format.

66.  trait FromRecord[T] extends Serializable

    Converts from an Avro GenericRecord into instances of T.

67.  case class ImmutableRecord(schema: Schema, values: Seq[AnyRef]) extends Record with Product with Serializable
68.  trait MagnoliaDerivedDecoders extends AnyRef
69.  trait MagnoliaDerivedEncoders extends AnyRef
70.  trait MagnoliaDerivedSchemaFors extends AnyRef
71.  case class NameExtractor(typeInfo: TypeInfo) extends Product with Serializable

    Extracts name and namespace from a TypeName.

    Extracts name and namespace from a TypeName. Takes into consideration provided annotations.

72.  trait Record extends GenericRecord with SpecificRecord

    An implementation of org.apache.avro.generic.GenericContainer that is both a GenericRecord and a SpecificRecord.

73.  class RecordDecoder[T] extends Decoder[T]
74.  class RecordEncoder[T] extends Encoder[T]
75.  trait RecordFormat[T] extends ToRecord[T] with FromRecord[T] with Serializable

    Brings together ToRecord and FromRecord in a single interface.

76.  trait ResolvableDecoder[T] extends Decoder[T]

    A Decoder that needs to be resolved before it is usable.

    A Decoder that needs to be resolved before it is usable. Resolution is needed to properly setup Decoder instances for recursive types.

    If this instance is used without resolution, it falls back to use an adhoc-resolved instance and delegates all operations to it. This involves a performance penalty of lazy val access that can be avoided by calling Encoder.resolveEncoder and using that.

    For examples on how to define custom ResolvableDecoder instances, see the Readme and RecursiveDecoderTest.

    T

    type this encoder is for (primitive type, case class, sealed trait, or enum e.g.).

77.  trait ResolvableEncoder[T] extends Encoder[T]

    An Encoder that needs to be resolved before it is usable.

    An Encoder that needs to be resolved before it is usable. Resolution is needed to properly setup Encoder instances for recursive types.

    If this instance is used without resolution, it falls back to use an adhoc-resolved instance and delegates all operations to it. This involves a performance penalty of lazy val access that can be avoided by calling Encoder.resolveEncoder and using that.

    For examples on how to define custom ResolvableEncoder instances, see the Readme and RecursiveEncoderTest.

    T

    type this encoder is for (primitive type, case class, sealed trait, or enum e.g.).

78.  trait ResolvableSchemaFor[T] extends SchemaFor[T]

    A SchemaFor that needs to be resolved before it is usable.

    A SchemaFor that needs to be resolved before it is usable. Resolution is needed to properly setup SchemaFor instances for recursive types.

    If this instance is used without resolution, it falls back to use an adhoc-resolved instance and delegates all operations to it. This involves a performance penalty of lazy val access that can be avoided by calling SchemaFor.resolveSchemaFor and using that.

    For examples on how to define custom ResolvableSchemaFor instances, see the Readme and RecursiveSchemaTest.

    T

    type this schema is for (primitive type, case class, sealed trait, or enum e.g.).

79.  case class ScalePrecision(scale: Int, precision: Int) extends Product with Serializable
80.  trait SchemaAware[+Typeclass[_], T] extends AnyRef

    Indicates that this typeclass (Codec, Encoder, Decoder) has a schema and is modifiable through schema changes

    Indicates that this typeclass (Codec, Encoder, Decoder) has a schema and is modifiable through schema changes

    Typeclass

    type extending SchemaAware (Codec, Encoder, Decoder)

    T

    data type this codec/encoder/decoder transforms from/to

81.  trait SchemaFor[T] extends Serializable

    A SchemaFor generates an Avro Schema for a Scala or Java type.

    A SchemaFor generates an Avro Schema for a Scala or Java type.

    For example, a String SchemaFor could return an instance of Schema.Type.STRING or Schema.Type.FIXED depending on the type required for Strings.

82.  sealed trait SchemaUpdate extends AnyRef
83.  sealed trait SealedTraitShape extends AnyRef
84.  trait ShapelessCoproductDecoders extends AnyRef
85.  trait ShapelessCoproductEncoders extends AnyRef
86.  trait ShapelessCoproductSchemaFors extends AnyRef
87.  trait TemporalDecoders extends AnyRef
88.  trait TemporalEncoders extends AnyRef
89.  trait ToRecord[T] extends Serializable

    Converts from instances of T into Record's.

    Converts from instances of T into Record's.

    Note: This interface requires that T is marshalled to an Avro GenericRecord / SpecificRecord, and therefore is limited to use by case classes or traits. This interface is essentially just a convenience wrapper around an Encoder so you do not have to cast the result.

    If you wish to convert an avro type other than record use an Encoder directly.

90.  trait TupleDecoders extends AnyRef
91.  trait TupleEncoders extends AnyRef
92.  trait TupleSchemaFors extends AnyRef
93.  trait TypeGuardedDecoding[T] extends Serializable
94.  case class TypeInfo(owner: String, short: String, typeArguments: Seq[TypeInfo], nameAnnotation: Option[String], namespaceAnnotation: Option[String], erased: Boolean) extends Product with Serializable
95.  class TypeUnionDecoder[T] extends Decoder[T]
96.  class TypeUnionEncoder[T] extends Encoder[T]
97.  class ValueTypeDecoder[T] extends Decoder[T]
98.  class ValueTypeEncoder[T] extends Encoder[T]