Builder

Value Members

1. final def !=(arg0: AnyRef): Boolean

   Definition Classes

   AnyRef

2. final def !=(arg0: Any): Boolean

   Definition Classes

   Any

3. final def ##(): Int

   Definition Classes

   AnyRef → Any

4. final def ==(arg0: AnyRef): Boolean

   Definition Classes

   AnyRef

5. final def ==(arg0: Any): Boolean

   Definition Classes

   Any

6. final def asInstanceOf[T0]: T0

   Definition Classes

   Any

7. def build(): KinesisInputDStream[Array[Byte]]

   Create a new instance of KinesisInputDStream with configured parameters and using the default message handler, which returns Array[Byte].

   Create a new instance of KinesisInputDStream with configured parameters and using the default message handler, which returns Array[Byte].

   returns

   Instance of KinesisInputDStream constructed with configured parameters

8. def buildWithMessageHandler[T](handler: (Record) ⇒ T)(implicit arg0: ClassTag[T]): KinesisInputDStream[T]

   Create a new instance of KinesisInputDStream with configured parameters and the provided message handler.

   Create a new instance of KinesisInputDStream with configured parameters and the provided message handler.

   handler

   Function converting Record instances read by the KCL to DStream type T

   returns

   Instance of KinesisInputDStream constructed with configured parameters

9. def checkpointAppName(appName: String): Builder

   Sets the KCL application name to use when checkpointing state to DynamoDB.

   Sets the KCL application name to use when checkpointing state to DynamoDB. This is a required parameter.

   appName

   Value to use for the KCL app name (used when creating the DynamoDB checkpoint table and when writing metrics to CloudWatch)

   returns

   Reference to this KinesisInputDStream.Builder

10. def checkpointInterval(interval: Duration): Builder

    Sets how often the KCL application state is checkpointed to DynamoDB.

    Sets how often the KCL application state is checkpointed to DynamoDB. Defaults to the Spark Streaming batch interval if no custom value is specified.

    interval

    Duration specifying how often the KCL state should be checkpointed to DynamoDB.

    returns

    Reference to this KinesisInputDStream.Builder

11. def clone(): AnyRef

    Attributes

    protected[java.lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

12. def cloudWatchCredentials(credentials: SparkAWSCredentials): Builder

    Sets the SparkAWSCredentials to use for authenticating to the AWS CloudWatch endpoint.

    Sets the SparkAWSCredentials to use for authenticating to the AWS CloudWatch endpoint. Will use the same credentials used for AWS Kinesis if no custom value is set.

    credentials

    SparkAWSCredentials to use for CloudWatch authentication

13. def dynamoDBCredentials(credentials: SparkAWSCredentials): Builder

    Sets the SparkAWSCredentials to use for authenticating to the AWS DynamoDB endpoint.

    Sets the SparkAWSCredentials to use for authenticating to the AWS DynamoDB endpoint. Will use the same credentials used for AWS Kinesis if no custom value is set.

    credentials

    SparkAWSCredentials to use for DynamoDB authentication

14. def endpointUrl(url: String): Builder

    Sets the AWS Kinesis endpoint URL.

    Sets the AWS Kinesis endpoint URL. Defaults to "https://kinesis.us-east-1.amazonaws.com" if no custom value is specified

    url

    Kinesis endpoint URL to use

    returns

    Reference to this KinesisInputDStream.Builder

15. final def eq(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

16. def equals(arg0: Any): Boolean

    Definition Classes

    AnyRef → Any

17. def finalize(): Unit

    Attributes

    protected[java.lang]

    Definition Classes

    AnyRef

    Annotations

    @throws( classOf[java.lang.Throwable] )

18. final def getClass(): Class[_]

    Definition Classes

    AnyRef → Any

19. def hashCode(): Int

    Definition Classes

    AnyRef → Any

20. def initialPositionInStream(initialPosition: InitialPositionInStream): Builder

    Sets the initial position data is read from in the Kinesis stream.

    Sets the initial position data is read from in the Kinesis stream. Defaults to InitialPositionInStream.LATEST if no custom value is specified.

    initialPosition

    InitialPositionInStream value specifying where Spark Streaming will start reading records in the Kinesis stream from

    returns

    Reference to this KinesisInputDStream.Builder

21. final def isInstanceOf[T0]: Boolean

    Definition Classes

    Any

22. def kinesisCredentials(credentials: SparkAWSCredentials): Builder

    Sets the SparkAWSCredentials to use for authenticating to the AWS Kinesis endpoint.

    Sets the SparkAWSCredentials to use for authenticating to the AWS Kinesis endpoint. Defaults to DefaultCredentialsProvider if no custom value is specified.

    credentials

    SparkAWSCredentials to use for Kinesis authentication

23. final def ne(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

24. final def notify(): Unit

    Definition Classes

    AnyRef

25. final def notifyAll(): Unit

    Definition Classes

    AnyRef

26. def regionName(regionName: String): Builder

    Sets the AWS region to construct clients for.

    Sets the AWS region to construct clients for. Defaults to "us-east-1" if no custom value is specified.

    regionName

    Name of AWS region to use (e.g. "us-west-2")

    returns

    Reference to this KinesisInputDStream.Builder

27. def storageLevel(storageLevel: StorageLevel): Builder

    Sets the storage level of the blocks for the DStream created.

    Sets the storage level of the blocks for the DStream created. Defaults to StorageLevel.MEMORY_AND_DISK_2 if no custom value is specified.

    storageLevel

    StorageLevel to use for the DStream data blocks

    returns

    Reference to this KinesisInputDStream.Builder

28. def streamName(streamName: String): Builder

    Sets the name of the Kinesis stream that the DStream will read from.

    Sets the name of the Kinesis stream that the DStream will read from. This is a required parameter.

    streamName

    Name of Kinesis stream that the DStream will read from

    returns

    Reference to this KinesisInputDStream.Builder

29. def streamingContext(jssc: JavaStreamingContext): Builder

    Sets the StreamingContext that will be used to construct the Kinesis DStream.

    Sets the StreamingContext that will be used to construct the Kinesis DStream. This is a required parameter.

    jssc

    JavaStreamingContext used to construct Kinesis DStreams

    returns

    Reference to this KinesisInputDStream.Builder

30. def streamingContext(ssc: StreamingContext): Builder

    Sets the StreamingContext that will be used to construct the Kinesis DStream.

    Sets the StreamingContext that will be used to construct the Kinesis DStream. This is a required parameter.

    ssc

    StreamingContext used to construct Kinesis DStreams

    returns

    Reference to this KinesisInputDStream.Builder

31. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes

    AnyRef

32. def toString(): String

    Definition Classes

    AnyRef → Any

33. final def wait(): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

34. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

35. final def wait(arg0: Long): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )