Job Instances Created from Templates

When you create job instances from a job template, you determine whether the job instances are attached to or detached from the parent job template. You also determine whether the completed job instance remains in the Job Instances view, or whether Control Hub automatically deletes the job instance when it completes.

When the pipeline included in the job template uses runtime parameters, you can create multiple job instances at the same time, specifying different parameter values for each instance. You also specify a suffix to append to the job name, so that each job instance uses a unique name.

When the pipeline included in the job template does not use runtime parameters, you can create a single job instance at a time.

Attached and Detached Job Instances

By default, each job instance is attached to the parent job template. When attached to the parent job template, the following statements are true:

Edits to the parent job template are applied to attached job instances
When you edit the parent job template, Control Hub applies the same updates to all attached job instances. If any attached job instances are currently running, then Control Hub restarts those instances to apply the updates to them.
For details about how Control Hub handles edits to parameter values in the job template, see Updating Job Template Parameters.
Attached job instances display in the parent job template's run history
When you view the details of the parent job template, Control Hub displays all currently running attached job instances in the Running Job Instances section and displays the run history of all attached job instances in the Run History section.
Attached job instances cannot be edited
You cannot directly edit attached job instances.

When you create a job instance, you can optionally detach the job instance from the parent job template. Detached job instances are not updated when the parent job template is edited, do not display in the parent job template details, and can be directly edited. Create a detached job instance when you want to use the job details and default parameter values defined in the job template, but don't want subsequent changes to the job template to be applied to the job instance. Instead, you want to be able to directly edit the detached job instance.

To detach a job instance, clear the Attach Instance to Template advanced option when you create the job instance.

Job Instance Completion

By default, the Job Instances view displays all job instances created from a pipeline and created from a job template, until a user deletes the job instances.

When you create a job instance from a job template, you can optionally configure Control Hub to automatically delete the job instance from the Job Instances view when the job finishes and transitions to an inactive status. You might want to delete job instances when completed to avoid listing a large number of inactive job instances in the Job Instances view.

When Control Hub automatically deletes attached job instances when completed, you can still view the attached job instances in the parent job template's run history.

To automatically delete completed job instances, select the Delete from Job Instances List when Completed advanced option when you create the job instance.

Suffix for Job Instance Names

When the pipeline included in the job template uses runtime parameters, you specify a suffix to append to the job name, so that each job instance uses a unique name.

The suffix is added to the job name after a hyphen, as follows:

<job  name> - <suffix>
Select one of the following methods to generate the suffix:
Counter
Control Hub appends a number to the job name. For example:
  • Web Log Collection Job - 1
  • Web Log Collection Job - 2
Timestamp
Control Hub appends a timestamp indicating when the job instance is started to the job name. For example:
  • Web Log Collection Job - 2021-10-22
  • Web Log Collection Job - 2021-10-23
Use a timestamp for the suffix when you plan to create and start job instances from the template at different times. For example, if you plan to start a single job instance every 24 hours. If you start multiple job instances from the template at the same time, Control Hub appends the same timestamp to each job instance name.
Parameter Value
Control Hub appends the value of the specified parameter to the job name. For example:
  • Web Log Collection Job - /server1/logs
  • Web Log Collection Job - /server2/logs

Parameter Values for Job Instances

When the pipeline included in the job template uses runtime parameters, you define unique parameter values for each job instance.

You can use functions included in the StreamSets expression language to define parameter values for each job instance.

Note: A job instance created from a Data Collector job template can run multiple pipeline instances, based on the number of pipeline instances configured for the job template. When a job instance runs multiple pipeline instances, each of those pipeline instances uses the same parameter values.

Use one of the following methods to define the parameter values when you create job instances:

Simple edit
Enter each parameter value in the provided field in the Create Job Instances wizard. Static parameters display read-only values that cannot be edited.
For example, the following simple edit configuration creates two job instances using different parameter values for the bucket parameter. The errordir parameter is static and cannot be edited:
Bulk edit
Enter parameter values in JSON format in the Create Job Instances wizard. When you add another job instance, Control Hub adds another group containing all the runtime parameters defined for the pipeline with the default values. Modify the parameter values, but be sure not to modify the parameter names.
For example, the following bulk edit configuration creates two job instances using different parameter values:

From file
Define the parameter values in JSON format in a text file and then select the file to upload to Control Hub in the Create Job Instances wizard. Each group of parameter names and values creates another job instance. Configuring parameter values in a file can be particularly useful when you have a large number of parameter values to enter.