Class MongoContinuousStream
- java.lang.Object
-
- com.mongodb.spark.sql.connector.read.MongoContinuousStream
-
- All Implemented Interfaces:
ContinuousStream,SparkDataStream
public class MongoContinuousStream extends java.lang.Object implements ContinuousStream
MongoContinuousStream defines how to read a stream of data from MongoDB.Utilizes MongoDBs change stream functionality, the continuous streams will consist of change events.
Note: Requires MongoDB 4.2+ To support continuing a change stream after a collection has been dropped.
-
-
Constructor Summary
Constructors Constructor Description MongoContinuousStream(StructType schema, ReadConfig readConfig)Construct a new instance
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description voidcommit(Offset end)ContinuousPartitionReaderFactorycreateContinuousReaderFactory()OffsetdeserializeOffset(java.lang.String json)OffsetinitialOffset()OffsetmergeOffsets(PartitionOffset[] offsets)InputPartition[]planInputPartitions(Offset start)voidstop()-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.spark.sql.connector.read.streaming.ContinuousStream
needsReconfiguration
-
-
-
-
Constructor Detail
-
MongoContinuousStream
public MongoContinuousStream(StructType schema, ReadConfig readConfig)
Construct a new instance- Parameters:
schema- the schema for the datareadConfig- the read configuration
-
-
Method Detail
-
planInputPartitions
public InputPartition[] planInputPartitions(Offset start)
- Specified by:
planInputPartitionsin interfaceContinuousStream
-
createContinuousReaderFactory
public ContinuousPartitionReaderFactory createContinuousReaderFactory()
- Specified by:
createContinuousReaderFactoryin interfaceContinuousStream
-
mergeOffsets
public Offset mergeOffsets(PartitionOffset[] offsets)
- Specified by:
mergeOffsetsin interfaceContinuousStream
-
initialOffset
public Offset initialOffset()
- Specified by:
initialOffsetin interfaceSparkDataStream
-
deserializeOffset
public Offset deserializeOffset(java.lang.String json)
- Specified by:
deserializeOffsetin interfaceSparkDataStream
-
commit
public void commit(Offset end)
- Specified by:
commitin interfaceSparkDataStream
-
stop
public void stop()
- Specified by:
stopin interfaceSparkDataStream
-
-