Object

com.audienceproject.aws.kinesis

PBScalaKinesisWriter

Related Doc: package kinesis

Permalink

object PBScalaKinesisWriter

This class can be used for writing aggregated Kinesis Stream Records to an Amazon Kinesis Stream. The Kinesis Stream Records are made of several User Records. This is a concept called aggregation in the context of Amazon Kinesis. This class only works with User Records that are Protocol Buffer based and their class is generated with the help of the ScalaPB sbt plugin (http://trueaccord.github.io/ScalaPB/).

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. PBScalaKinesisWriter
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  5. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  6. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  10. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  11. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  12. val logger: Logger

    Permalink
  13. val maximumLastSize: Int

    Permalink

    The maximum size in bytes of the last User Record that we can add to the Stream Record.

    The maximum size in bytes of the last User Record that we can add to the Stream Record. This is very empiric becuase we don't have access to the size function of the aggregated record.

  14. val maximumRetries: Int

    Permalink

    The maximum number of linear back-off retries before giving up and throwing an exception

  15. val maximumSize: Int

    Permalink

    The maximum size of the aggregated record in bytes before the last record is added.

    The maximum size of the aggregated record in bytes before the last record is added. Look above for explanation.

  16. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  17. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  18. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  19. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  20. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  21. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  22. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  23. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  24. def write(streamName: String, it: Iterator[GeneratedMessage], client: AmazonKinesis, raygunClient: RaygunClient): Int

    Permalink

    Send an iterator of Protocol Buffer encoded messages to Kinesis.

    Send an iterator of Protocol Buffer encoded messages to Kinesis. It also uses a RaygunClient. Raygun is a 3rd party error logging service provider.

    Example:

    val it = List(
        new PBMessage("now"),
        new PBMessage("yesterday"),
        new PBMessage("tomorrow")
    ).toIterator
    
    val client = new AmazonKinesisClient(new ProfileCredentialsProvider("my-custom-profile"))
    
    val raygunClient = new RaygunClient("your-raygunClient-key")
    
    PBScalaKinesisWriter.write("test-stream", it, client, raygunClient)
    streamName

    The name of the Kinesis Stream where the data should go to

    it

    The iterator containing Protocol Buffers messages

    client

    The Kinesis client responsible for sending the data to the Kinesis Streams

    raygunClient

    The Raygun client which sends exceptions

  25. def write(streamName: String, it: Iterator[GeneratedMessage], raygunClient: RaygunClient): Int

    Permalink

    Send an iterator of Protocol Buffer encoded messages to Kinesis.

    Send an iterator of Protocol Buffer encoded messages to Kinesis. It uses a default Kinesis client built using DefaultAWSCredentialsProviderChain and the default region. It also uses a RaygunClient. Raygun is a 3rd party error logging service provider.

    Example:

    val it = List(
        new PBMessage("now"),
        new PBMessage("yesterday"),
        new PBMessage("tomorrow")
    ).toIterator
    
    val raygunClient = new RaygunClient("your-raygunClient-key")
    
    PBScalaKinesisWriter.write("test-stream", it, raygunClient)
    streamName

    The name of the Kinesis Stream where the data should go to

    it

    The iterator containing Protocol Buffers messages

    raygunClient

    The Raygun client which sends exceptions

  26. def write(streamName: String, it: Iterator[GeneratedMessage], client: AmazonKinesis): Int

    Permalink

    Send an iterator of Protocol Buffer encoded messages to Kinesis.

    Send an iterator of Protocol Buffer encoded messages to Kinesis.

    Example:

    val it = List(
        new PBMessage("now"),
        new PBMessage("yesterday"),
        new PBMessage("tomorrow")
    ).toIterator
    
    val client = new AmazonKinesisClient(new ProfileCredentialsProvider("my-custom-profile"))
    
    PBScalaKinesisWriter.write("test-stream", it, client)
    streamName

    The name of the Kinesis Stream where the data should go to

    it

    The iterator containing Protocol Buffers messages

    client

    The Kinesis client responsible for sending the data to the Kinesis Streams

  27. def write(streamName: String, it: Iterator[GeneratedMessage]): Int

    Permalink

    Send an iterator of Protocol Buffer encoded messages to Kinesis.

    Send an iterator of Protocol Buffer encoded messages to Kinesis. It uses a default Kinesis client built using DefaultAWSCredentialsProviderChain and the default region.

    Example:

    val it = List(
        new PBMessage("now"),
        new PBMessage("yesterday"),
        new PBMessage("tomorrow")
    ).toIterator
    
    PBScalaKinesisWriter.write("test-stream", it)
    streamName

    The name of the Kinesis Stream where the data should go to

    it

    The iterator containing Protocol Buffers messages

Inherited from AnyRef

Inherited from Any

Ungrouped