Package org.apache.pulsar.functions.runtime

Class RuntimeUtils

  • public class RuntimeUtils
extends java.lang.Object
    Util class for common runtime functionality.

    • Constructor Summary

      Constructors 
      Constructor Description
      RuntimeUtils()  

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static java.util.List<java.lang.String> composeCmd​(org.apache.pulsar.functions.instance.InstanceConfig instanceConfig, java.lang.String instanceFile, java.lang.String extraDependenciesDir, java.lang.String logDirectory, java.lang.String originalCodeFileName, java.lang.String pulsarServiceUrl, java.lang.String stateStorageServiceUrl, org.apache.pulsar.functions.instance.AuthenticationConfig authConfig, java.lang.String shardId, java.lang.Integer grpcPort, java.lang.Long expectedHealthCheckInterval, java.lang.String logConfigFile, java.lang.String secretsProviderClassName, java.lang.String secretsProviderConfig, java.lang.Boolean installUserCodeDependencies, java.lang.String pythonDependencyRepository, java.lang.String pythonExtraDependencyRepository, java.lang.String narExtractionDirectory, java.lang.String functionInstanceClassPath, java.lang.String pulsarWebServiceUrl)  
      static java.lang.String genFunctionLogFolder​(java.lang.String logDirectory, org.apache.pulsar.functions.instance.InstanceConfig instanceConfig)  
      static java.util.List<java.lang.String> getArgsBeforeCmd​(org.apache.pulsar.functions.instance.InstanceConfig instanceConfig, java.lang.String extraDependenciesDir)  
      static java.util.List<java.lang.String> getCmd​(org.apache.pulsar.functions.instance.InstanceConfig instanceConfig, java.lang.String instanceFile, java.lang.String extraDependenciesDir, java.lang.String logDirectory, java.lang.String originalCodeFileName, java.lang.String pulsarServiceUrl, java.lang.String stateStorageServiceUrl, org.apache.pulsar.functions.instance.AuthenticationConfig authConfig, java.lang.String shardId, java.lang.Integer grpcPort, java.lang.Long expectedHealthCheckInterval, java.lang.String logConfigFile, java.lang.String secretsProviderClassName, java.lang.String secretsProviderConfig, java.lang.Boolean installUserCodeDependencies, java.lang.String pythonDependencyRepository, java.lang.String pythonExtraDependencyRepository, java.lang.String narExtractionDirectory, java.lang.String functionInstanceClassPath, boolean k8sRuntime, java.lang.String pulsarWebServiceUrl)  
      static java.util.List<java.lang.String> getGoInstanceCmd​(org.apache.pulsar.functions.instance.InstanceConfig instanceConfig, java.lang.String originalCodeFileName, java.lang.String pulsarServiceUrl, boolean k8sRuntime)
      Different from python and java function, Go function uploads a complete executable file(including: instance file + user code file).
      static java.lang.String getPrometheusMetrics​(int metricsPort)  
      static <T> T getRuntimeFunctionConfig​(java.util.Map<java.lang.String,​java.lang.Object> configMap, java.lang.Class<T> functionRuntimeConfigClass)  
      static void registerDefaultCollectors​(org.apache.pulsar.functions.instance.stats.FunctionCollectorRegistry registry)  
      static java.lang.String[] splitRuntimeArgs​(java.lang.String input)
      Regex for splitting a string using space when not surrounded by single or double quotes

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • FUNCTIONS_INSTANCE_CLASSPATH

        public static final java.lang.String FUNCTIONS_INSTANCE_CLASSPATH
        See Also:
        Constant Field Values

    • Constructor Detail

      • RuntimeUtils

        public RuntimeUtils()

    • Method Detail

      • composeCmd

        public static java.util.List<java.lang.String> composeCmd​(org.apache.pulsar.functions.instance.InstanceConfig instanceConfig,
                                                          java.lang.String instanceFile,
                                                          java.lang.String extraDependenciesDir,
                                                          java.lang.String logDirectory,
                                                          java.lang.String originalCodeFileName,
                                                          java.lang.String pulsarServiceUrl,
                                                          java.lang.String stateStorageServiceUrl,
                                                          org.apache.pulsar.functions.instance.AuthenticationConfig authConfig,
                                                          java.lang.String shardId,
                                                          java.lang.Integer grpcPort,
                                                          java.lang.Long expectedHealthCheckInterval,
                                                          java.lang.String logConfigFile,
                                                          java.lang.String secretsProviderClassName,
                                                          java.lang.String secretsProviderConfig,
                                                          java.lang.Boolean installUserCodeDependencies,
                                                          java.lang.String pythonDependencyRepository,
                                                          java.lang.String pythonExtraDependencyRepository,
                                                          java.lang.String narExtractionDirectory,
                                                          java.lang.String functionInstanceClassPath,
                                                          java.lang.String pulsarWebServiceUrl)
                                                   throws java.lang.Exception
        Throws:
        java.lang.Exception

      • getArgsBeforeCmd

        public static java.util.List<java.lang.String> getArgsBeforeCmd​(org.apache.pulsar.functions.instance.InstanceConfig instanceConfig,
                                                                java.lang.String extraDependenciesDir)

      • getGoInstanceCmd

        public static java.util.List<java.lang.String> getGoInstanceCmd​(org.apache.pulsar.functions.instance.InstanceConfig instanceConfig,
                                                                java.lang.String originalCodeFileName,
                                                                java.lang.String pulsarServiceUrl,
                                                                boolean k8sRuntime)
                                                         throws java.io.IOException
        Different from python and java function, Go function uploads a complete executable file(including: instance file + user code file). Its parameter list is provided to the broker in the form of a yaml file, the advantage of this approach is that backward compatibility is guaranteed. In Java and Python the instance is managed by broker (or function worker) so the changes in command line is under control; but in Go the instance is compiled with the user function, so pulsar doesn't have the control what instance is used in the function. Hence in order to support BC for go function, we can't dynamically add more commandline arguments. Using an instance config to pass the parameters from function worker to go instance is the best way for maintaining the BC.

        When we run the go function, we only need to specify the location of the go-function file and the yaml file. The content of the yaml file will be automatically generated according to the content provided by instanceConfig.

        Throws:
        java.io.IOException

      • getCmd

        public static java.util.List<java.lang.String> getCmd​(org.apache.pulsar.functions.instance.InstanceConfig instanceConfig,
                                                      java.lang.String instanceFile,
                                                      java.lang.String extraDependenciesDir,
                                                      java.lang.String logDirectory,
                                                      java.lang.String originalCodeFileName,
                                                      java.lang.String pulsarServiceUrl,
                                                      java.lang.String stateStorageServiceUrl,
                                                      org.apache.pulsar.functions.instance.AuthenticationConfig authConfig,
                                                      java.lang.String shardId,
                                                      java.lang.Integer grpcPort,
                                                      java.lang.Long expectedHealthCheckInterval,
                                                      java.lang.String logConfigFile,
                                                      java.lang.String secretsProviderClassName,
                                                      java.lang.String secretsProviderConfig,
                                                      java.lang.Boolean installUserCodeDependencies,
                                                      java.lang.String pythonDependencyRepository,
                                                      java.lang.String pythonExtraDependencyRepository,
                                                      java.lang.String narExtractionDirectory,
                                                      java.lang.String functionInstanceClassPath,
                                                      boolean k8sRuntime,
                                                      java.lang.String pulsarWebServiceUrl)
                                               throws java.lang.Exception
        Throws:
        java.lang.Exception

      • genFunctionLogFolder

        public static java.lang.String genFunctionLogFolder​(java.lang.String logDirectory,
                                                    org.apache.pulsar.functions.instance.InstanceConfig instanceConfig)

      • getPrometheusMetrics

        public static java.lang.String getPrometheusMetrics​(int metricsPort)
                                             throws java.io.IOException
        Throws:
        java.io.IOException

      • splitRuntimeArgs

        public static java.lang.String[] splitRuntimeArgs​(java.lang.String input)
        Regex for splitting a string using space when not surrounded by single or double quotes

      • getRuntimeFunctionConfig

        public static <T> T getRuntimeFunctionConfig​(java.util.Map<java.lang.String,​java.lang.Object> configMap,
                                             java.lang.Class<T> functionRuntimeConfigClass)

      • registerDefaultCollectors

        public static void registerDefaultCollectors​(org.apache.pulsar.functions.instance.stats.FunctionCollectorRegistry registry)