org.apache.beam.sdk.util.construction.resources

Class PipelineResources

java.lang.Object

org.apache.beam.sdk.util.construction.resources.PipelineResources

public class PipelineResources
extends java.lang.Object

Utilities for working with classpath resources for pipelines.

Constructor Summary

Constructors

Constructor and Description
PipelineResources()

Method Summary

Modifier and Type	Method and Description
static java.util.List<java.lang.String>	detectClassPathResourcesToStage(java.lang.ClassLoader classLoader, PipelineOptions options) Uses algorithm provided via PipelineResourcesOptions to detect classpath resources.
static void	prepareFilesForStaging(FileStagingOptions options) Goes through the list of files that need to be staged on runner.
static java.util.List<java.lang.String>	prepareFilesForStaging(java.util.List<java.lang.String> resourcesToStage, java.lang.String tmpJarLocation) Goes through the list of files that need to be staged on runner.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PipelineResources

public PipelineResources()

Method Detail

detectClassPathResourcesToStage

public static java.util.List<java.lang.String> detectClassPathResourcesToStage(java.lang.ClassLoader classLoader,
                                                                               PipelineOptions options)

Uses algorithm provided via PipelineResourcesOptions to detect classpath resources.

Parameters:

classLoader - The URLClassLoader to use to detect resources to stage (optional).

options - pipeline options

Returns:

A list of absolute paths to the resources the class loader uses.

prepareFilesForStaging

public static void prepareFilesForStaging(FileStagingOptions options)

Goes through the list of files that need to be staged on runner. Removes nonexistent directories and packages existing ones. This is necessary for runners that require filesToStage to be jars only.

This method mutates the filesToStage value of the given options.

Parameters:

options - options object with the files to stage and temp location for staging

prepareFilesForStaging

public static java.util.List<java.lang.String> prepareFilesForStaging(java.util.List<java.lang.String> resourcesToStage,
                                                                      java.lang.String tmpJarLocation)

Goes through the list of files that need to be staged on runner. Fails if the directory does not exist and packages the ones that exist. This is necessary for runners that require filesToStage to be jars only.

Parameters:

resourcesToStage - list of resources that need to be staged

tmpJarLocation - temporary directory to store the jars

Returns:

A list of absolute paths to resources (jar files)

Throws:

java.lang.IllegalStateException - if the directory to be staged does not exist