Package org.apache.flink.table.connector.sink

Interface DataStreamSinkProvider

  • All Superinterfaces:
    org.apache.flink.table.connector.sink.DynamicTableSink.SinkRuntimeProvider, org.apache.flink.table.connector.ParallelismProvider
    @PublicEvolving
public interface DataStreamSinkProvider
extends org.apache.flink.table.connector.sink.DynamicTableSink.SinkRuntimeProvider, org.apache.flink.table.connector.ParallelismProvider
    Provider that consumes a Java DataStream as a runtime implementation for DynamicTableSink.

    Note: This provider is only meant for advanced connector developers. Usually, a sink should consist of a single entity expressed via SinkFunctionProvider, or OutputFormatProvider. When using a DataStream an implementer needs to pay attention to how changes are shuffled to not mess up the changelog per parallel subtask.

    • Method Summary

      All Methods Instance Methods Abstract Methods Default Methods 
      Modifier and Type Method Description
      org.apache.flink.streaming.api.datastream.DataStreamSink<?> consumeDataStream​(org.apache.flink.table.connector.ProviderContext providerContext, org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.table.data.RowData> dataStream)
      Consumes the given Java DataStream and returns the sink transformation DataStreamSink.
      default Optional<Integer> getParallelism()

    • Method Detail

      • consumeDataStream

        org.apache.flink.streaming.api.datastream.DataStreamSink<?> consumeDataStream​(org.apache.flink.table.connector.ProviderContext providerContext,
                                                                              org.apache.flink.streaming.api.datastream.DataStream<org.apache.flink.table.data.RowData> dataStream)
        Consumes the given Java DataStream and returns the sink transformation DataStreamSink.

        Note: If the CompiledPlan feature should be supported, this method MUST set a unique identifier for each transformation/operator in the data stream. This enables stateful Flink version upgrades for streaming jobs. The identifier is used to map state back from a savepoint to an actual operator in the topology. The framework can generate topology-wide unique identifiers with ProviderContext.generateUid(String).

        See Also:
        SingleOutputStreamOperator.uid(String)

      • getParallelism

        default Optional<Integer> getParallelism()

        Note: If a custom parallelism is returned and consumeDataStream(ProviderContext, DataStream) applies multiple transformations, make sure to set the same custom parallelism to each operator to not mess up the changelog.

        Specified by:
        getParallelism in interface org.apache.flink.table.connector.ParallelismProvider