Package io.pravega.connectors.flink.sink
Class PravegaEventWriter<T>
- java.lang.Object
-
- io.pravega.connectors.flink.sink.PravegaEventWriter<T>
-
- Type Parameters:
T
- The type of the event to be written.
- All Implemented Interfaces:
java.lang.AutoCloseable
,org.apache.flink.api.connector.sink2.SinkWriter<T>
public class PravegaEventWriter<T> extends java.lang.Object implements org.apache.flink.api.connector.sink2.SinkWriter<T>
A PravegaSinkWriter
implementation that is suitable forPravegaWriterMode.BEST_EFFORT
andPravegaWriterMode.ATLEAST_ONCE
.Note that the difference between these two modes is that
flushAndVerify()
is called for each checkpoint in thePravegaWriterMode.ATLEAST_ONCE
mode.
-
-
Field Summary
Fields Modifier and Type Field Description protected io.pravega.client.EventStreamClientFactory
clientFactory
protected java.util.concurrent.ExecutorService
executorService
protected java.util.concurrent.atomic.AtomicLong
pendingWritesCount
protected java.util.concurrent.atomic.AtomicReference<java.lang.Throwable>
writeError
-
Constructor Summary
Constructors Constructor Description PravegaEventWriter(org.apache.flink.api.connector.sink2.Sink.InitContext context, io.pravega.client.ClientConfig clientConfig, io.pravega.client.stream.Stream stream, PravegaWriterMode writerMode, org.apache.flink.api.common.serialization.SerializationSchema<T> serializationSchema, PravegaEventRouter<T> eventRouter)
A Pravega non-transactional writer that handlesPravegaWriterMode.BEST_EFFORT
andPravegaWriterMode.ATLEAST_ONCE
writer mode.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
close()
void
flush(boolean endOfInput)
void
flushAndVerify()
protected PravegaEventRouter<T>
getEventRouter()
protected io.pravega.client.stream.EventStreamWriter<T>
getInternalWriter()
protected PravegaWriterMode
getWriterMode()
protected io.pravega.client.stream.EventStreamWriter<T>
initializeInternalWriter()
void
write(T element, org.apache.flink.api.connector.sink2.SinkWriter.Context context)
-
-
-
Field Detail
-
writeError
@VisibleForTesting protected volatile java.util.concurrent.atomic.AtomicReference<java.lang.Throwable> writeError
-
pendingWritesCount
@VisibleForTesting protected java.util.concurrent.atomic.AtomicLong pendingWritesCount
-
executorService
@VisibleForTesting protected transient java.util.concurrent.ExecutorService executorService
-
clientFactory
@VisibleForTesting protected transient io.pravega.client.EventStreamClientFactory clientFactory
-
-
Constructor Detail
-
PravegaEventWriter
public PravegaEventWriter(org.apache.flink.api.connector.sink2.Sink.InitContext context, io.pravega.client.ClientConfig clientConfig, io.pravega.client.stream.Stream stream, PravegaWriterMode writerMode, org.apache.flink.api.common.serialization.SerializationSchema<T> serializationSchema, PravegaEventRouter<T> eventRouter)
A Pravega non-transactional writer that handlesPravegaWriterMode.BEST_EFFORT
andPravegaWriterMode.ATLEAST_ONCE
writer mode.- Parameters:
context
- Some runtime info from sink.clientConfig
- The Pravega client configuration.stream
- The destination stream.writerMode
- The Pravega writer mode.serializationSchema
- The implementation for serializing every event into pravega's storage format.eventRouter
- The implementation to extract the partition key from the event.
-
-
Method Detail
-
initializeInternalWriter
@VisibleForTesting protected io.pravega.client.stream.EventStreamWriter<T> initializeInternalWriter()
-
write
public void write(T element, org.apache.flink.api.connector.sink2.SinkWriter.Context context) throws java.io.IOException, java.lang.InterruptedException
- Specified by:
write
in interfaceorg.apache.flink.api.connector.sink2.SinkWriter<T>
- Throws:
java.io.IOException
java.lang.InterruptedException
-
flush
public void flush(boolean endOfInput) throws java.io.IOException, java.lang.InterruptedException
- Specified by:
flush
in interfaceorg.apache.flink.api.connector.sink2.SinkWriter<T>
- Throws:
java.io.IOException
java.lang.InterruptedException
-
flushAndVerify
@VisibleForTesting public void flushAndVerify() throws java.io.IOException, java.lang.InterruptedException
- Throws:
java.io.IOException
java.lang.InterruptedException
-
close
public void close() throws java.lang.Exception
- Specified by:
close
in interfacejava.lang.AutoCloseable
- Throws:
java.lang.Exception
-
getWriterMode
@VisibleForTesting protected PravegaWriterMode getWriterMode()
-
getEventRouter
@VisibleForTesting @Nullable protected PravegaEventRouter<T> getEventRouter()
-
getInternalWriter
@VisibleForTesting protected io.pravega.client.stream.EventStreamWriter<T> getInternalWriter()
-
-