Interface SplitHintSpec

    • Method Detail

      • split

        <T> Iterator<List<T>> split​(Iterator<T> inputIterator,
                                    Function<T,​InputFileAttribute> inputAttributeExtractor)
        Returns an iterator of splits. A split has a list of files of the type SplitHintSpec.
        Parameters:
        inputIterator - that returns input files.
        inputAttributeExtractor - to create InputFileAttribute for each input file. This may involve a network call, so implementations of SplitHintSpec should use it only if needed, and reuse results if appropriate.