Class FilePerSplitHintSpec

  • All Implemented Interfaces:
    SplitHintSpec

    public class FilePerSplitHintSpec
    extends Object
    implements SplitHintSpec
    Assigns each input file to its own split. Not named as a JsonSubTypes in SplitHintSpec, because this class is meant for internal use within a server only. It is not serialized for transfer between servers, and is not part of the user-facing API.
    • Method Detail

      • split

        public <T> Iterator<List<T>> split​(Iterator<T> inputIterator,
                                           Function<T,​InputFileAttribute> inputAttributeExtractor)
        Description copied from interface: SplitHintSpec
        Returns an iterator of splits. A split has a list of files of the type SplitHintSpec.
        Specified by:
        split in interface SplitHintSpec
        Parameters:
        inputIterator - that returns input files.
        inputAttributeExtractor - to create InputFileAttribute for each input file. This may involve a network call, so implementations of SplitHintSpec should use it only if needed, and reuse results if appropriate.