com.sksamuel.avro4s

An implementation of AvroInputStream that reads values of type T written as binary data.

In avro, binary encodings do not include the schema. Therefore, this input stream requires that the user provide the schema.

A Decoder must be provided (usually implicitly) that will marshall avro records into instances of type T.

An implementation of AvroInputStream that reads values of type T written as Avro.

Avro data files contain the schema as part of the message payload. Therefore, no schema is necessarily required to read the data back and the decoder will use the schema present in the payload. However, for efficiency, if the schema is provided, then a decoder can be pre-built and used on each contained object.

A Decoder must be provided (usually implicitly) that will marshall avro records into instances of type T.

An AvroOutputStream that writes the schema along with the messages.

This is usually the format required when writing multiple messages to a single file.

Some frameworks, such as a Kafka, store the Schema separately to messages, in which case the AvroBinaryInputStream can be used.

This annotation is used to disable generics in the encoding of a record's name.

Normally, the record name for a generic type is the name of the raw type, plus the actual type parameters. For example, a class Foo with type parameters Int and Boolean, would have a generated name of Foo__Int_Boolean

When this annotation is present on a type, the name used in the schema will simply be the raw type, eg Foo.

Use this annotation to indicate that a record type should be generated as an avro error type.

AvroFixed overrides the schema type for a field or a value class so that the schema is set to org.apache.avro.Schema.Type.FIXED rather than whatever the default would be.

This annotation can be used in the following ways:

• On a field, eg case class Foo(@AvroField(10) name: String) which results in the field name having schema type FIXED with a size of 10.

• On a value type, eg @AvroField(7) case class Foo(name: String) extends AnyVal which results in all usages of the value type having schema FIXED with a size of 7 rather than the default.

AvroName allows the name used by Avro to be different from what is defined in code.

For example, if a case class defines a field z, such as case class Foo(z: String) then normally this will be serialized as an entry 'z' in the Avro Record.

However, if the field is annotated such as case class Foo(@AvroName("x") z: String) then the entry in the Avro Record will be for 'x'.

Similarly for deserialization, if a field is annotated then the name that is looked up in the avro record will be the annotated name and not the field name in Scala.

The second example is with classes. If a class is annotated with @AvroName then the name used in the record schema will not be the classname but the annotated value.

This will also have an effect on serialization. For example, when decoding records into an Either, the decoder must decide if the value is a Left or a Right. It usually does this by comparing the name in the record to the classnames of the either types, but when annotated, it will compare the name in the record to the annotated value.

An AvroOutputStream will write instances of T to an underlying representation.

There are three implementations of this stream

• a Data stream,
• a Binary stream
• a Json stream

See the methods on the companion object to create instances of each of these types of stream.

Creates an Avro Schema for an arbitrary type T.

This type is called AvroSchema and not just Schema to facilitate easy importing when mixing with org.apache.avro.Schema.

Convenience trait that allows to define both custom Encoder and custom Decoder in one go, if both are needed. This way, they don't have to be declared in two separate classes or objects.

A codec can both encode a Scala value of type T into a compatible Avro value based on the given schema as well as decode an Avro value, such as a GenericRecord, SpecificRecord, GenericFixed, EnumSymbol, or a basic JVM type, into a target Scala type.

A Decoder is used to convert an Avro value, such as a GenericRecord, SpecificRecord, GenericFixed, EnumSymbol, or a basic type, into a specified Scala type.

For example, a Decoder[String] would convert an input into a plain Java String.

Another example, a decoder for Option[String] would handle inputs of null by emitting a None, and a non-null input by emitting a String wrapped in a Some.

An Encoder encodes a Scala value of type T into a JVM value suitable for use with Avro.

For example, an encoder could encode a String as an instance of Utf8, or it could encode it as an instance of GenericFixed.

Alternatively, given a Scala enum value, the enum could be encoded as an instance of GenericData.EnumSymbol or as a String.

An encoder is invoked with an Avro schema, and a FieldMapper and returns a reusable function that then encodes values of type T into Avro types.

It is possible to configure encoders entirely through annotations, which is fine if your system is self contained. But if your schemas are generated outside of avro4s, or even in another language, you may need to use these "third-party" schemas to influence the encoding process.

Some encoders use the schema to determine the encoding function to return. For example, strings can be encoded as UTF8s, GenericFixeds, ByteBuffers or java.lang.Strings. Therefore the Encoder[String] typeclass instances uses the schema to select which of these implementations to use.

Other types may not require the schema at all. For example, the default Encoder[Int] always returns a java.lang.Integer regardless of any schema input.

The second parameter to an encoder is the field mapper. This is used to derive the field names used when generating record or error types. By default, the field mapper will use the field names as they are defined in the type itself (from the case class). However for interop with other systems you may wish to customize this, for example, by writing out field names in snake_case or adding a prefix.

We may have a schema with a field in snake case like say { "first_name": "sam" } and that schema needs to be used for a case class field firstName.

The FieldMapper is used to map fields in a schema to fields in a class by transforming the class field name to the wire name format.

Converts from an Avro IndexedRecord into instances of T.

An implementation of org.apache.avro.generic.GenericContainer that is both a GenericRecord and a SpecificRecord.

Brings together ToRecord and FromRecord in a single interface.

Returns a RecordFormat that will convert to/from instances of T and avro Record's.

A SchemaFor generates an Avro Schema for a Scala or Java type.

For example, a SchemaFor[String] could return a schema of type Schema.Type.STRING, and a SchemaFor[Int] could return an schema of type Schema.Type.INT

Converts from instances of T into Record's.

Note: This interface requires that T is marshalled to an Avro GenericRecord / SpecificRecord, and therefore is limited to use by case classes or traits. This interface is essentially just a convenience wrapper around an Encoder so you do not have to cast the result.

If you wish to convert an avro type other than record use an Encoder directly.