Job Templates

When you create a job for a pipeline that uses runtime parameters, you can enable the job to work as a job template. A job template lets you run multiple job instances with different runtime parameter values from a single job definition.

Like a regular job, a job instance can run multiple pipeline instances, based on the number of pipeline instances configured for the job template. When a job instance runs multiple pipeline instances, each of those pipeline instances uses the same parameter values.

For example, you design a pipeline with a Google Cloud Storage origin that uses a runtime parameter for the Bucket property for the origin. You create a job template for the pipeline and configure the job template to run one pipeline instance. You create and start five job instances from the template, defining a unique value for the bucket parameter for each job instance. Control Hub starts five active jobs from the single job template, running one pipeline instance for each job. Each job and pipeline instance has an identical configuration, except for the bucket parameter value.

Control Hub lists job templates in the Jobs view, identifying each template with a Job Template label in the Status column. A job template is simply a job definition and does not have an active or inactive status. When you start a job template, you create and start one or more job instances from the template. You specify a suffix to uniquely name each job instance, the number of job instances to create from the template, and the parameter values to use for each job instance.

For example, the following image displays a job template, identified with the Job Template label in the Status column, and two job instances started from that template. The job instance names use a counter as the suffix:

You manage job instances started from a template just as you manage any other job. For example, you can individually start, stop, and monitor job instances.

However, you cannot edit inactive job instances started from a template. Instead, you must edit the job template to edit the job definition. Active job instances reflect edits made to the job template after you stop and restart the job instances.