com.tuplejump.kafka.connect.cassandra
Reads in the user provided configuration for a given source or sink.
Reads in the user provided configuration for a given source or sink.
Fails fast on org.apache.kafka.connect.connector.Connector start if not valid.
Because user params can be a mine field on bugs farther down stream we attempt to catch as many up front as we can.
the user config
if requirements not met
Returns the FQCN of the partitioner, which will NOT be on the classpath.
Returns the FQCN of the partitioner, which will NOT be on the classpath. org.apache.cassandra.dht.{Murmur3Partitioner, RandomPartitioner...}
Returns a list of records when available by polling this SourceTask for new records.
Returns a list of records when available by polling this SourceTask for new records. From the kafka doc: "This method should block if no data is currently available."
Initial implementation only supports bulk load for a query.
Restarts call stop() then start().
Restarts call stop() then start().
Returns table metadata if the keyspace and table exist in the Cassandra cluster being connected to, and the coordinates have been configured.
Returns table metadata if the keyspace and table exist in the Cassandra cluster being connected to, and the coordinates have been configured.
The datastax java driver returns null Keyspace or null Table
if either do not exist, so we alert the user application via
ConnectException and do not proceed, to not propagate NPEs.
The parsed values based on the config map passed in on start.
The parsed values based on the config map passed in on start. The Kafka mutable java map is converted to a scala immutable, parsed based on whether this is for a source or sink or task thereof, then validated by type.
If a source task, this will have one or more SourceConfigs. If a sink task, this will have one or more SinkConfigs. Currently we hand each task a unique topic to keyspace and table mapping. Roadmap: Parallism partitioning strategy configurable by user.
Initialized as the default, user can override.
Initialized as the default, user can override.
A Cassandra
SourceTaskrun by a KafkaWorkerSourceTask. In a Cassandra source task, the query may be selected column reads of pre-aggregated data vs reflect all columns in the table schema.