Field Detail

• public static final java.lang.String TASK_CONTEXT_PROPERTY_NAME

• protected boolean allScalarValues

• protected boolean blocking

  Whenever the process execution is required to be blocking in order to handle shared object in a thread safe manner

• protected boolean completed

  Flag set true when the processor termination has been invoked See #checkProcessTermination

• protected ProcessConfig config

  The corresponding task configuration properties, it holds the inputs/outputs definition as well as other execution meta-declaration

• protected final java.lang.ThreadLocal<TaskRun> currentTask

  Keeps track of the task instance executed by the current thread

• protected Executor executor

  The underlying executor which will run the task

• protected ch.grengine.Grengine grengine

  Groovy engine used to evaluate dynamic code

• protected groovyx.gpars.group.PGroup group

  Gpars thread pool

• protected boolean hasEachParams

• protected final java.lang.Object indexCount

  Unique task index number (run)

• protected java.lang.String name

  The processor descriptive name

• protected java.util.concurrent.atomic.AtomicIntegerArray openPorts

  Track the status of input ports. When 1 the port is open (waiting for data), when 0 the port is closed (ie. received the STOP signal)

• protected groovyx.gpars.dataflow.operator.DataflowProcessor operator

  The corresponding DataflowProcessor which will receive and manage accordingly the task inputs note: it must be declared volatile -- issue #41

• protected BaseScript ownerScript

  The script object which defines this task

• protected Session session

  The current workflow execution session

• protected boolean singleton

  Whenever the process is executed only once

• protected groovyx.gpars.agent.Agent<StateObj> state

  The state is maintained by using an agent

• protected TaskBody taskBody

  The piece of code to be execute provided by the user

Property Detail

• static java.util.Random RND

• static java.util.List tips

Constructor Detail

• protected TaskProcessor()

• TaskProcessor(java.lang.String name, Executor executor, Session session, BaseScript script, ProcessConfig config, TaskBody taskBody)

  Create and initialize the processor object

  Parameters:

  name

  executor

  session

  script

  config

  taskBody

Method Detail

• static java.lang.String bashEnvironmentScript(java.util.Map<java.lang.String, java.lang.String> environment, boolean escape = false )

  Given a map holding variables key-value pairs, create a script fragment exporting the required environment variables

• protected void bindOutParam(OutParam param, java.util.List values)

• protected void bindOutputs(TaskRun task)

  Bind the expected output files to the corresponding output channels

  Parameters:

  processor

• protected final boolean checkCachedOrLaunchTask(TaskRun task, com.google.common.hash.HashCode hash, boolean shouldTryCache)

  Try to check if exists a previously executed process result in the a cached folder. If it exists use the that result and skip the process execution, otherwise the task is sumitted for execution.

  Parameters:

  task - The TaskRun instance to be executed

  hash - The unique HashCode for the given task inputs

  script - The script to be run (only when it's a merge task)

  Returns:

  false when a cached result has been found and the execution has skipped, or true if the task has been submitted for execution

• final boolean checkCachedOutput(TaskRun task, java.nio.file.Path folder, com.google.common.hash.HashCode hash)

  Check whenever the outputs for the specified task already exist

  Parameters:

  task - The task instance

  folder - The folder where the outputs are stored (eventually)

  Returns:

  true when all outputs are available, false otherwise

• protected ErrorStrategy checkErrorStrategy(TaskRun task, ProcessException error, int taskErrCount, int procErrCount)

• final boolean checkStoredOutput(TaskRun task)

  Check if exists a *storeDir* for the specified task. When if exists and contains the expected result files, the process execution is skipped.

  Parameters:

  task - The task for which check the stored output

  Returns:

  true when the folder exists and it contains the expected outputs, false otherwise

• protected boolean checkWhenGuard(TaskRun task)

• protected void collectOutFiles(TaskRun task, FileOutParam param, java.nio.file.Path workDir, java.util.Map context)

• protected void collectOutValues(TaskRun task, ValueOutParam param, java.util.Map ctx)

• protected void collectOutputs(TaskRun task)

• protected final void collectOutputs(TaskRun task, java.nio.file.Path workDir, java.lang.Object stdout, java.util.Map context)

  Once the task has completed this method is invoked to collected all the task results

  Parameters:

  task

• protected void collectStdOut(TaskRun task, StdOutParam param, java.lang.Object stdout)

  Collects the process 'std output'

  Parameters:

  task - The executed process instance

  param - The declared StdOutParam object

  stdout - The object holding the task produced std out object

• protected void createOperator()

  Verify if this process run only one time

• protected final com.google.common.hash.HashCode createTaskHashKey(TaskRun task)

• protected final TaskRun createTaskRun()

  Create a new TaskRun instance, initializing the following properties :

• TaskRun#id
• TaskRun#status
• TaskRun#index
• TaskRun#name
• TaskRun#process

  Returns:

  The new newly created TaskRun

• java.lang.String dumpTerminationStatus()

  Dump the current process status listing all input *port* statuses for debugging purpose

  Returns:

  The text description representing the process status

• @groovy.transform.CompileStatic protected java.util.List<nextflow.file.FileHolder> expandWildcards(java.lang.String name, java.util.List<nextflow.file.FileHolder> files)

  An input file name may contain wildcards characters which have to be handled coherently given the number of files specified.

  Parameters:

  name - A file name with may contain a wildcard character star * or question mark ?. Only one occurrence can be specified for star or question mark wildcards.

  value - Any value that have to be managed as an input files. Values other than Path are converted to a string value, using the #toString method and saved in the local file-system. Value of type Collection are expanded to multiple values accordingly.

  Returns:

• @groovy.transform.CompileStatic protected java.lang.String expandWildcards0(java.lang.String path, java.lang.String stageName, int index, int size)

• static java.lang.String fetchInterpreter(java.lang.String script)

  Given the task script extract the top *she-bang* interpreter declaration removing the #! characters.

  Parameters:

  script - The script to be executed

  Returns:

  The interpreter as defined in the she-bang declaration, for example /usr/bin/env perl

• protected final java.util.List<java.lang.String> formatGuardError(java.util.List<java.lang.String> message, FailedGuardException error, TaskRun task)

• protected final java.util.List<java.lang.String> formatTaskError(java.util.List<java.lang.String> message, java.lang.Throwable error, TaskRun task)

• ProcessConfig getConfig()

  Returns:

  The TaskConfig object holding the task configuration properties

• Executor getExecutor()

  Returns:

  The Executor associated to this processor

• int getId()

  Returns:

  The processor unique id

• java.lang.String getName()

  Returns:

  The processor name

• groovyx.gpars.dataflow.operator.DataflowProcessor getOperator()

  Returns:

  The DataflowOperator underlying this process

• BaseScript getOwnerScript()

  Returns:

  The BaseScript object which represents pipeline script

• @groovy.transform.Memoized java.util.Map<java.lang.String, java.lang.String> getProcessEnvironment()

  Returns:

  The map holding the shell environment variables for the task to be executed

• protected java.lang.String getRndTip()

  Display a random tip at the bottom of the error report

  Returns:

  The tip string to display

• protected ScriptType getScriptType()

  Define the type of script hold by the #code property

• Session getSession()

  Returns:

  The current Session instance

• @groovy.transform.Memoized protected java.util.List<java.nio.file.Path> getTaskBinEntries(java.lang.String script)

  This method scans the task command string looking for invocations of scripts defined in the project bin folder.

  Parameters:

  script - The task command string

  Returns:

  The list of paths of scripts in the project bin folder referenced in the task command

• TaskBody getTaskBody()

  Returns:

  The user provided script block

• protected java.util.Map<java.lang.String, java.lang.Object> getTaskGlobalVars(TaskRun task)

• protected final java.util.Map<java.lang.String, java.lang.Object> getTaskGlobalVars(java.util.Set<java.lang.String> variableNames, groovy.lang.Binding binding, java.util.Map context)

  Parameters:

  variableNames - The collection of variables referenced in the task script

  binding - The script global binding

  context - The task variable context

  Returns:

  The set of task variables accessed in global script context and not declared as input/output

• protected final boolean handleException(java.lang.Throwable error, TaskRun task = null )

  Handles an error raised during the processor execution

  Parameters:

  error - The exception raised during the task execution

  task - The TaskDef instance which raised the exception

  Returns:

  true to terminate the processor execution, false ignore the error and continue to process other pending tasks

• protected final void invokeTask(java.lang.Object args)

  The processor execution body

  Parameters:

  processor

  values

• boolean isCacheable()

  Whenever the process can be cached

• protected final int makeTaskContextStage1(TaskRun task, java.util.Map secondPass, java.util.List values)

• protected final void makeTaskContextStage2(TaskRun task, java.util.Map secondPass, int count)

• protected final void makeTaskContextStage3(TaskRun task, com.google.common.hash.HashCode hash, java.nio.file.Path folder)

• protected nextflow.file.FileHolder normalizeInputToFile(java.lang.Object input, java.lang.String altName)

  An input file parameter can be provided with any value other than a file. This function normalize a generic value to a Path create a temporary file in the for it.

  Parameters:

  input - The input value

  altName - The name to be used when a temporary file is created.

  Returns:

  The Path that will be staged in the task working folder

• protected java.util.List<nextflow.file.FileHolder> normalizeInputToFiles(java.lang.Object obj, int count)

• static java.lang.String normalizeScript(java.lang.String script, java.lang.Object shell)

  Remove extra leading, trailing whitespace and newlines chars, also if the script does not start with a shebang line, add the default by using the current #shell attribute

• @groovy.transform.CompileStatic protected void publishOutputs(TaskRun task)

  Publish output files to a specified target folder

  Parameters:

  task - The task whose outputs need to be published

  overwrite - When true any existing file will be overwritten, otherwise the publishing is ignored

• @groovy.transform.CompileStatic protected java.lang.String replaceQuestionMarkWildcards(java.lang.String name, int index)

• @groovy.transform.CompileStatic protected java.lang.String replaceStarWildcards(java.lang.String name, int index, boolean strip = false)

• java.lang.Object run()

  Launch the 'script' define by the code closure as a local bash script

  Parameters:

  code - A Closure retuning a bash script e.g.

                {
                   """
                   #!/bin/bash
                   do this \${x}
                   do that \${y}
                   :
                   """
                }

  Returns:

  this instance

• protected final void sendPoisonPill()

  Send a poison pill over all the outputs channel

• static java.lang.Object shebangLine(java.lang.Object shell)

  Returns:

  A string 'she-bang' formatted to the added on top script to be executed. The interpreter to be used define bu the *taskConfig* property shell

• protected java.lang.Object singleItemOrList(java.util.List<nextflow.file.FileHolder> items)

• protected final void submitTask(TaskRun task, com.google.common.hash.HashCode hash, java.nio.file.Path folder)

  Execute the specified task shell script

  Parameters:

  script - The script string to be execute, e.g. a BASH script

  Returns:

  TaskDef

• protected void terminateProcess()

• protected void validateInputSets(java.util.List values)