Package org.apache.flink.python.util

Class PythonDependencyUtils

java.lang.Object

org.apache.flink.python.util.PythonDependencyUtils

@Internal
public class PythonDependencyUtils
extends Object

Utility class for Python dependency management. The dependencies will be registered at the distributed cache.

Field Summary

Fields

Modifier and Type	Field	Description
static String	CACHE
static String	FILE
static String	FILE_DELIMITER
static String	PARAM_DELIMITER

Constructor Summary

Constructors

Constructor	Description
PythonDependencyUtils()

Method Summary

Modifier and Type	Method	Description
static org.apache.flink.configuration.Configuration	configurePythonDependencies(org.apache.flink.configuration.ReadableConfig config)	Adds python dependencies to registered cache file list according to given configuration and returns a new configuration which contains the metadata of the registered python dependencies.
static String	getArchiveTargetDirName(String archiveValue)	Extracts the target directory name from an archive value string.
static void	merge(org.apache.flink.configuration.Configuration config, org.apache.flink.configuration.Configuration pythonConfiguration)
static org.apache.flink.configuration.Configuration	parsePythonDependencyConfiguration(org.apache.commons.cli.CommandLine commandLine)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

FILE

public static final String FILE

See Also:

Constant Field Values

CACHE

public static final String CACHE

See Also:

Constant Field Values

FILE_DELIMITER

public static final String FILE_DELIMITER

See Also:

Constant Field Values

PARAM_DELIMITER

public static final String PARAM_DELIMITER

See Also:

Constant Field Values

Constructor Detail

PythonDependencyUtils

public PythonDependencyUtils()

Method Detail

getArchiveTargetDirName

public static String getArchiveTargetDirName(String archiveValue)

Extracts the target directory name from an archive value string.

The archive value may contain a PARAM_DELIMITER separator. If present, the format is "originalFileName#targetDirName" and the target directory name is the part after the delimiter. Otherwise, the entire value is used as the target directory name.

Parameters:

archiveValue - the archive value string, e.g. "venv.zip#venv" or "venv"

Returns:

the target directory name

configurePythonDependencies

public static org.apache.flink.configuration.Configuration configurePythonDependencies(org.apache.flink.configuration.ReadableConfig config)

Adds python dependencies to registered cache file list according to given configuration and returns a new configuration which contains the metadata of the registered python dependencies.

Parameters:

config - The configuration which contains python dependency configuration.

Returns:

A new configuration which contains the metadata of the registered python dependencies.

parsePythonDependencyConfiguration

public static org.apache.flink.configuration.Configuration parsePythonDependencyConfiguration(org.apache.commons.cli.CommandLine commandLine)

merge

public static void merge(org.apache.flink.configuration.Configuration config,
                         org.apache.flink.configuration.Configuration pythonConfiguration)