Create an input stream that pulls message from a Aliyun DTS stream.
Create an input stream that pulls message from a Aliyun DTS stream.
JavaStreamingContext jssc = ...; String accessKeyId = "kj7aY*******UYx6"; String accessKeySecret = "AiNMAlxz*************1PxaPaL8t"; String guid = "dts-guid-name"; JavaReceiverInputDStream<String> dtsStream = DtsUtils.createStream( ssc, accessKeyId, accessKeySecret, guid, StorageLevel.MEMORY_AND_DISK_2, false); dtsStream.foreachRDD(rdd => { ... });
Java streamingContext object.
Aliyun Access Key ID.
Aliyun Access Key Secret.
Aliyun DTS guid name.
Storage level to use for storing the received objects. StorageLevel.MEMORY_AND_DISK_2 is recommended.
use public ip or not.
Create an input stream that pulls message from a Aliyun DTS stream.
Create an input stream that pulls message from a Aliyun DTS stream.
JavaStreamingContext jssc = ...; String accessKeyId = "kj7aY*******UYx6"; String accessKeySecret = "AiNMAlxz*************1PxaPaL8t"; String guid = "dts-guid-name"; static class ReadMessage implements Function<ClusterMessage, String> { @Override public String call(ClusterMessage msg) { return msg.getRecord.toString; } } JavaReceiverInputDStream<String> dtsStream = DtsUtils.createStream( ssc, accessKeyId, accessKeySecret, guid, ReadMessage, StorageLevel.MEMORY_AND_DISK_2, false); dtsStream.foreachRDD(rdd => { ... });
Java streamingContext object.
Aliyun Access Key ID.
Aliyun Access Key Secret.
Aliyun DTS guid name.
Extract information from DTS record message.
Storage level to use for storing the received objects. StorageLevel.MEMORY_AND_DISK_2 is recommended.
use public ip or not.
Create an input stream that pulls message from a Aliyun DTS stream.
Create an input stream that pulls message from a Aliyun DTS stream.
val ssc: StreamingSparkContext = ... val accessKeyId = "kj7aY*******UYx6" val accessKeySecret = "AiNMAlxz*************1PxaPaL8t" val guid = "dts-guid-name" val dtsStream = DtsUtils.createStream( ssc, accessKeyId, accessKeySecret, guid, StorageLevel.MEMORY_AND_DISK_2, false) dtsStream.foreachRDD(rdd => { ... })
StreamingContext object.
Aliyun Access Key ID.
Aliyun Access Key Secret.
Aliyun DTS guid name.
Storage level to use for storing the received objects. StorageLevel.MEMORY_AND_DISK_2 is recommended.
use public ip or not.
Create an input stream that pulls message from a Aliyun DTS stream.
Create an input stream that pulls message from a Aliyun DTS stream.
val ssc: StreamingSparkContext = ... val accessKeyId = "kj7aY*******UYx6" val accessKeySecret = "AiNMAlxz*************1PxaPaL8t" val guid = "dts-guid-name" def func: ClusterMessage => String = msg => msg.getRecord.toString val dtsStream = DtsUtils.createStream( ssc, accessKeyId, accessKeySecret, guid, func, StorageLevel.MEMORY_AND_DISK_2, false) dtsStream.foreachRDD(rdd => { ... })
StreamingContext object.
Aliyun Access Key ID.
Aliyun Access Key Secret.
Aliyun DTS guid name.
Extract information from DTS record message.
Storage level to use for storing the received objects. StorageLevel.MEMORY_AND_DISK_2 is recommended.
use public ip or not.
Various utility classes for working with Aliyun DTS.