Class MongoContinuousStream
- java.lang.Object
-
- com.mongodb.spark.sql.connector.read.MongoContinuousStream
-
- All Implemented Interfaces:
ContinuousStream
,SparkDataStream
public class MongoContinuousStream extends java.lang.Object implements ContinuousStream
MongoContinuousStream defines how to read a stream of data from MongoDB.Utilizes MongoDBs change stream functionality, the continuous streams will consist of change events.
Note: Requires MongoDB 4.2+ To support continuing a change stream after a collection has been dropped.
-
-
Constructor Summary
Constructors Constructor Description MongoContinuousStream(StructType schema, ReadConfig readConfig)
Construct a new instance
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
commit(Offset end)
ContinuousPartitionReaderFactory
createContinuousReaderFactory()
Offset
deserializeOffset(java.lang.String json)
Offset
initialOffset()
Offset
mergeOffsets(PartitionOffset[] offsets)
InputPartition[]
planInputPartitions(Offset start)
void
stop()
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.spark.sql.connector.read.streaming.ContinuousStream
needsReconfiguration
-
-
-
-
Constructor Detail
-
MongoContinuousStream
public MongoContinuousStream(StructType schema, ReadConfig readConfig)
Construct a new instance- Parameters:
schema
- the schema for the datareadConfig
- the read configuration
-
-
Method Detail
-
planInputPartitions
public InputPartition[] planInputPartitions(Offset start)
- Specified by:
planInputPartitions
in interfaceContinuousStream
-
createContinuousReaderFactory
public ContinuousPartitionReaderFactory createContinuousReaderFactory()
- Specified by:
createContinuousReaderFactory
in interfaceContinuousStream
-
mergeOffsets
public Offset mergeOffsets(PartitionOffset[] offsets)
- Specified by:
mergeOffsets
in interfaceContinuousStream
-
initialOffset
public Offset initialOffset()
- Specified by:
initialOffset
in interfaceSparkDataStream
-
deserializeOffset
public Offset deserializeOffset(java.lang.String json)
- Specified by:
deserializeOffset
in interfaceSparkDataStream
-
commit
public void commit(Offset end)
- Specified by:
commit
in interfaceSparkDataStream
-
stop
public void stop()
- Specified by:
stop
in interfaceSparkDataStream
-
-