Interface ParallelismProvider

  • All Known Subinterfaces:
    InputFormatProvider, OutputFormatProvider, SinkV2Provider, SourceProvider

    @PublicEvolving
    public interface ParallelismProvider
    Parallelism provider for other connector providers. It allows to express a custom parallelism for the connector runtime implementation. Otherwise, the parallelism is determined by the planner.
    • Method Detail

      • getParallelism

        default Optional<Integer> getParallelism()
        Returns the parallelism for this instance.

        The parallelism denotes how many parallel instances of a source or sink will be spawned during the execution.

        Enforcing a different parallelism for sources/sinks might mess up the changelog if the output/input is not ChangelogMode.insertOnly(). Therefore, a primary key is required by which the output/input will be shuffled after/before records leave/enter the ScanTableSource.ScanRuntimeProvider/DynamicTableSink.SinkRuntimeProvider implementation.

        Returns:
        empty if the connector does not provide a custom parallelism, then the planner will decide the number of parallel instances by itself.