Purpose
Some jobs share a need for the same set of task sequences. For this situation, a separate workflow containing the set of tasks can be created and the childJob task can be used to run the separate workflow. In addition to making workflows resuable, child jobs reduce the complexity of workflows by reducing them to a smaller set of childJob tasks. This improves the readability of workflows. The childJob task also enables users to run multiple iterations of that same workflow in parallel or sequentially.
Child Jobs With No Loop
The childJob task starts a job inside the currently running job. The calling or parent job waits for the child to finish (complete or cancel) before progressing to its next task. If the child job is cancelled, it will error and the job will progress along the childJob task's error transition. If it completes, job execution will progress along the childJob task's success transition.
Anything that is a job variable in the child job is placed in an object with a <job_variable__name> : <job_variable_value> key/value pair format. Variables can then be passed up to the parent childJob task via the outgoing schema of the child workflow. Then, if a particular variable is needed, a transformation can be used to query out that key (job variable name).
Child Jobs That Loop
Multiple iterations (loops) of the same child job can be run using a childJob task sequentially or in parallel by specifying the loopType of the childJob task to be sequential or parallel. An iteration of the child job is run for each element of the input Loop Array, so that the length of the array determines the total number of child jobs run. If run sequentially, each child job iteration is run after the previous child job iteration is finished (complete or cancelled). When run in parallel, all iterations are started at once, and the childJob task finishes after the final child job finishes.
If the childJob task loops and contains a transformation input, the provided transformation is run on each element of Loop Array to produce the job variables required prior to starting each child job. The result of each transformation run is passed into the corresponding child job iteration as job variables. The task allows the creation of transformations locally. If created locally on the task, the outgoing schema of the created transformation is set as the input schema of the child workflow. A single incoming schema is created by default with $id and currentValue. When the childJob task is run, each element of Loop Array is passed as currentValue in the transformation. If additional input schemas are added to the transformation besides currentValue, these additional schemas are added as additional JST parameters that can be passed in via the childJob task. This allows users to define specific values that do not change across iterations.
If the childJob task loops and contains a Loop Array input with no transformation, each element of the input Loop Array is passed into the corresponding child job iteration directly as input job variables. In this case, it is required that each element is type object, with the same <job_variable__name> : <job_variable_value> key/value pair format used with non-looping child jobs.
A childJob task that loops results in an error if all the child jobs are cancelled. If at least one child job is complete, the task ends in status complete. The job variables (determined by the output schema) and final status of each child job iteration is returned in the task output under the loop property.
When a parent job is cancelled which contains a childJob task that loops, all non-completed loop iterations will also be cancelled. Any loop iterations that have not yet been created, such as when the Loop Array has a large number of elements, will never run.
Properties
ChildJob properties are referenced in the table below.
| Property | Description |
|---|---|
| Actor | Set to Job to run the child job as the same user that started the parent.Set to Pronghorn for the child job to be run as the global Pronghorn user. |
| Workflow | Name of the child workflow to be started by the task. |
| Job Variables | When the child job requires job variables, those variables will be listed in the Job Variables section. The name of each job variable will be listed with Location and Value fields. Specify where to find values for each child's job variables in the Location dropdown. Values can come from a job variable, a static value, or an earlier task's outgoing variable. If the Location is a job variable or earlier task, the Value shows a dropdown of available variable names for selection. If the Location is static, the Value variable becomes a text edit box. |
| Loop Type | Specifies the type of loop. If the loop type is None, only one child job is started. Parallel starts multiple child jobs in parallel, and Sequential starts them sequentially (one at a time). |
| Loop Array | Array of job variables to pass into each child job iteration. Each element of the array must either match the child workflow's input schema or be transformed to match it for the child jobs to be started successfully. If a transformation is provided, the transformation is run on each element of Loop Array to create the job variables passed into each child job. ⚠ WARNING: The maximum array length size is 10,000 elements. In addition, care should be taken to avoid creating such large datasets, otherwise task execution could potentially stall within workflow engine. |
| Transformation (JST) | Transformation (JST) to run on each element of Loop Array. Each element of Loop Array is set to currentValue in the provided transformation. The result of each transformation run is passed into the corresponding child job as job variables and should match the child's input schema. |
ChildJob With No Loop Example
In the following example, the child workflow named child requires three (3) variables to start: ping, time, and pong. The Loop Type is set to None in the loop settings menu, so only one child job starts.
Task Details for No Loop
Pictured below are task details for the child job. The Workflow input is the name of the workflow the childJob task runs. The workflow can be edited by pressing on the Edit Workflow icon next to the name. The Refresh Workflow icon pulls the latest variables required to start the workflow, which should be done if new variables are required after the workflow is edited.
Figure 1: ChildJob Details
Job Variables for No Loop
Pictured below are the required job variables for the child workflow. Each of these can be set statically, via a job variable, or a prior task reference in the workflow.
Figure 2: ChildJob Variables
Child Job With No Loop Results
The following output is after the child job finishes with status complete. Variables that are passed into the output are determined by the output schema of the child job, which in this case includes the incoming job variables and the _id and initiator job variables.
Figure 3: No Loop Results
ChildJob With Loop And Transformation Example
In the following example, the child workflow named child is run with the Loop Type set to Parallel. The same task details are used as shown above in the non-looping child job case. The following walks through the creation of a transformation locally to run on each element of Loop Array.
Parallel Loop Settings
Below displays the Loop Settings prior to the addition of a transformation. If no transformation is provided, the child job will directly pass in each element of Loop Array as job variables to the child job. If any element is not type object, the childJob task will error. A new transformation can be created by pressing the Create New JST button to the right of the transformation. Existing transformations can also be used.
Figure 4: Loop Settings No Transformation
Local Transformation Creation Default
Below displays the default transformation created when the Create New JST option is selected with name childTransformation. Notice that the outgoing schemas by default are set to the three job variables required by child to start.
Figure 5: Default Transformation
Local Transformation Creation Continued
Assume that each value of Loop Array includes only the pong integer and assume the time and ping inputs do not depend on Loop Array (do not change between iterations). The below shows what the transformation would look like in this case. Note that currentValue corresponds to each element of the Loop Array. If Loop Array contains object elements, JST methods can be used to transform and query out the required job variables and convert them to the required form if needed. Notice that two (2) addditional incoming schemas were added, timeInput and pingInput. Once saved, these are required as additional JST params that can be set on the childJob task below the transformation.
Figure 6: Transformation Set
Post Creation of Local Transformation
Once the locally created transformation is saved, the Refresh JST button can be clicked to get the latest JST parameters. These can then be set statically or by job or task reference. Below shows the Loop Settings post-transformation refresh. Notice that the additional incoming schemas each have the two (2) additional JST params (timeInput and pingInput) that can be set. Once set, these are passed to the JST as a fixed value between each child job iteration.
Figure 7: JST Parameters
Results After Complete Child Job Task
The following shows the results of the above created child job task in the case that all child jobs are finished and at least one child completes. The Loop input determines which loop's variables will be displayed. The Max element indicates how many loops were run during the child job excecution.
Figure 8: Loop Result
Results With Cancelled Child Jobs
Below displays the case that all but one of the child jobs are cancelled. Notice that the task still succeeds.
Figure 9: Cancelled Result
All Child Jobs Cancelled
Pictured below is the error message when all child jobs are cancelled.
Figure 10: Error Result
ChildJob With Loop And No Transformation Example
If the childJob task loops, but contains no transformation, the elements of the Loop Array are passed in directly as job variables to the child job. Below displays an example of this with a static Loop Array providing all required job variables for the workflow named child. Note that transformations are the preferred route to take for child job looping.
Figure 11: Loop No Transformation
Job Details Page Navigation
To navigate the child jobs on the job details page, the childJob task can be expanded to display all child jobs started by the task. The eyeball icon can then be clicked to navigate to the job details page for that child job.
Figure 12: Job Details ChildJob Loop
